Thijs/wiki.techinc.nl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
wiki.techinc.nl/tests/phpunit/languages/utils/CLDRPluralRuleEvaluatorTest.php

152 lines
6.3 KiB
PHP
Raw Normal View History

CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
<?php
/**
* @author Niklas Laxström
* @file
*/
@covers tags for the rest of test files.. Change-Id: I0fafe80531325a412472ab7c9fc6d81c861b3751
2013-10-24 20:30:43 +00:00
/**
* @covers CLDRPluralRuleEvaluator
*/
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
class CLDRPluralRuleEvaluatorTest extends MediaWikiTestCase {
/**
* @dataProvider validTestCases
*/
function testValidRules( $expected, $rules, $number, $comment ) {
pass codesniffer on tests/ Fix almost all occurences of the following sniffs: Generic.CodeAnalysis.UselessOverridingMethod.Found Generic.Formatting.NoSpaceAfterCast.SpaceFound Generic.Functions.FunctionCallArgumentSpacing.SpaceBeforeComma Generic.Functions.OpeningFunctionBraceKernighanRitchie.BraceOnNewLine Generic.PHP.LowerCaseConstant.Found PSR2.Classes.PropertyDeclaration.ScopeMissing PSR2.Files.EndFileNewline.TooMany PSR2.Methods.MethodDeclaration.StaticBeforeVisibility Change-Id: I96aacef5bafe5a2bca659744fba1380999cfc37d
2013-01-28 10:27:15 +00:00
$result = CLDRPluralRuleEvaluator::evaluate( $number, (array)$rules );
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
$this->assertEquals( $expected, $result, $comment );
}
/**
* @dataProvider invalidTestCases
* @expectedException CLDRPluralRuleError
*/
function testInvalidRules( $rules, $comment ) {
pass codesniffer on tests/ Fix almost all occurences of the following sniffs: Generic.CodeAnalysis.UselessOverridingMethod.Found Generic.Formatting.NoSpaceAfterCast.SpaceFound Generic.Functions.FunctionCallArgumentSpacing.SpaceBeforeComma Generic.Functions.OpeningFunctionBraceKernighanRitchie.BraceOnNewLine Generic.PHP.LowerCaseConstant.Found PSR2.Classes.PropertyDeclaration.ScopeMissing PSR2.Files.EndFileNewline.TooMany PSR2.Methods.MethodDeclaration.StaticBeforeVisibility Change-Id: I96aacef5bafe5a2bca659744fba1380999cfc37d
2013-01-28 10:27:15 +00:00
CLDRPluralRuleEvaluator::evaluate( 1, (array)$rules );
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
}
function validTestCases() {
$tests = array(
Improving CLDR Plural Rule Evaluator documentation. Change-Id: Ic6581de7cc69dea7af0bee5596497db509d6b9ba
2013-11-22 00:42:30 +00:00
# expected, rule, number, comment
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
array( 0, 'n is 1', 1, 'integer number and is' ),
array( 0, 'n is 1', "1", 'string integer number and is' ),
array( 0, 'n is 1', 1.0, 'float number and is' ),
array( 0, 'n is 1', "1.0", 'string float number and is' ),
array( 1, 'n is 1', 1.1, 'float number and is' ),
array( 1, 'n is 1', 2, 'float number and is' ),
Update formatting 8 of n. Change-Id: I55551510e7afde5b6b981697d5c0efd7b9507585
2013-02-15 10:35:55 +00:00
array( 0, 'n in 1,3,5', 3, '' ),
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
array( 1, 'n not in 1,3,5', 5, '' ),
Update formatting 8 of n. Change-Id: I55551510e7afde5b6b981697d5c0efd7b9507585
2013-02-15 10:35:55 +00:00
array( 1, 'n in 1,3,5', 2, '' ),
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
array( 0, 'n not in 1,3,5', 4, '' ),
Update formatting 8 of n. Change-Id: I55551510e7afde5b6b981697d5c0efd7b9507585
2013-02-15 10:35:55 +00:00
array( 0, 'n in 1..3', 2, '' ),
array( 0, 'n in 1..3', 3, 'in is inclusive' ),
array( 1, 'n in 1..3', 0, '' ),
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
Update formatting 8 of n. Change-Id: I55551510e7afde5b6b981697d5c0efd7b9507585
2013-02-15 10:35:55 +00:00
array( 1, 'n not in 1..3', 2, '' ),
array( 1, 'n not in 1..3', 3, 'in is inclusive' ),
array( 0, 'n not in 1..3', 0, '' ),
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
array( 1, 'n is not 1 and n is not 2 and n is not 3', 1, 'and relation' ),
array( 0, 'n is not 1 and n is not 2 and n is not 4', 3, 'and relation' ),
array( 0, 'n is not 1 or n is 1', 1, 'or relation' ),
array( 1, 'n is 1 or n is 2', 3, 'or relation' ),
array( 0, 'n is 1', 1, 'extra whitespace' ),
array( 0, 'n mod 3 is 1', 7, 'mod' ),
array( 0, 'n mod 3 is not 1', 4.3, 'mod with floats' ),
array( 0, 'n within 1..3', 2, 'within with integer' ),
array( 0, 'n within 1..3', 2.5, 'within with float' ),
array( 0, 'n in 1..3', 2, 'in with integer' ),
array( 1, 'n in 1..3', 2.5, 'in with float' ),
array( 0, 'n in 3 or n is 4 and n is 5', 3, 'and binds more tightly than or' ),
array( 1, 'n is 3 or n is 4 and n is 5', 4, 'and binds more tightly than or' ),
array( 0, 'n mod 10 in 3..4,9 and n mod 100 not in 10..19,70..79,90..99', 24, 'breton rule' ),
array( 1, 'n mod 10 in 3..4,9 and n mod 100 not in 10..19,70..79,90..99', 25, 'breton rule' ),
array( 0, 'n within 0..2 and n is not 2', 0, 'french rule' ),
array( 0, 'n within 0..2 and n is not 2', 1, 'french rule' ),
array( 0, 'n within 0..2 and n is not 2', 1.2, 'french rule' ),
array( 1, 'n within 0..2 and n is not 2', 2, 'french rule' ),
array( 1, 'n in 3..10,13..19', 2, 'scottish rule - ranges with comma' ),
array( 0, 'n in 3..10,13..19', 4, 'scottish rule - ranges with comma' ),
array( 1, 'n in 3..10,13..19', 12.999, 'scottish rule - ranges with comma' ),
array( 0, 'n in 3..10,13..19', 13, 'scottish rule - ranges with comma' ),
array( 0, '5 mod 3 is n', 2, 'n as result of mod - no need to pass' ),
Plural rules: updates for UTS #35 Rev 33 * New operands i, v, w, f, t * New operators =, !=, % * Ignore "samples", which are basically unit tests embedded in rule specifications * Ignore the new "other" rules, which have an empty condition. It doesn't really makes sense to parse them, since the empty condition means special handling should be done in the caller, it is not equivalent to an unconditional true or false. * Trailing zero support requires that the input number be a string. Documented this. * Fixed some comments * Added test cases for new features Bug: 56931 Change-Id: I96986c0c664f785e75b0a4ced2ec9e37b72681c1
2013-12-11 23:45:07 +00:00
# Revision 33 new operand examples
# expected, rule, number, comment
array( 0, 'i is 1', '1.00', 'new operand i' ),
array( 0, 'v is 2', '1.00', 'new operand v' ),
array( 0, 'w is 0', '1.00', 'new operand w' ),
array( 0, 'f is 0', '1.00', 'new operand f' ),
array( 0, 't is 0', '1.00', 'new operand t' ),
array( 0, 'i is 1', '1.30', 'new operand i' ),
array( 0, 'v is 2', '1.30', 'new operand v' ),
array( 0, 'w is 1', '1.30', 'new operand w' ),
array( 0, 'f is 30', '1.30', 'new operand f' ),
array( 0, 't is 3', '1.30', 'new operand t' ),
array( 0, 'i is 1', '1.03', 'new operand i' ),
array( 0, 'v is 2', '1.03', 'new operand v' ),
array( 0, 'w is 2', '1.03', 'new operand w' ),
array( 0, 'f is 3', '1.03', 'new operand f' ),
array( 0, 't is 3', '1.03', 'new operand t' ),
# Revision 33 new operator aliases
# expected, rule, number, comment
array( 0, 'n % 3 is 1', 7, 'new % operator' ),
array( 0, 'n = 1,3,5', 3, 'new = operator' ),
array( 1, 'n != 1,3,5', 5, 'new != operator' ),
# Revision 33 samples
# expected, rule, number, comment
Pass phpcs-strict on some test files (11/11) Woo! Change-Id: I9fc116dfdf18c2772d047adb5bb14535d0bd39ed
2014-04-24 17:52:51 +00:00
// @codingStandardsIgnoreStart Ignore Generic.Files.LineLength.TooLong
Plural rules: updates for UTS #35 Rev 33 * New operands i, v, w, f, t * New operators =, !=, % * Ignore "samples", which are basically unit tests embedded in rule specifications * Ignore the new "other" rules, which have an empty condition. It doesn't really makes sense to parse them, since the empty condition means special handling should be done in the caller, it is not equivalent to an unconditional true or false. * Trailing zero support requires that the input number be a string. Documented this. * Fixed some comments * Added test cases for new features Bug: 56931 Change-Id: I96986c0c664f785e75b0a4ced2ec9e37b72681c1
2013-12-11 23:45:07 +00:00
array( 0, 'n in 1,3,5@integer 3~10, 103~110, 1003, … @decimal 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 103.0, 1003.0, …', 3, 'samples' ),
Pass phpcs-strict on some test files (11/11) Woo! Change-Id: I9fc116dfdf18c2772d047adb5bb14535d0bd39ed
2014-04-24 17:52:51 +00:00
// @codingStandardsIgnoreEnd
Plural rules: updates for UTS #35 Rev 33 * New operands i, v, w, f, t * New operators =, !=, % * Ignore "samples", which are basically unit tests embedded in rule specifications * Ignore the new "other" rules, which have an empty condition. It doesn't really makes sense to parse them, since the empty condition means special handling should be done in the caller, it is not equivalent to an unconditional true or false. * Trailing zero support requires that the input number be a string. Documented this. * Fixed some comments * Added test cases for new features Bug: 56931 Change-Id: I96986c0c664f785e75b0a4ced2ec9e37b72681c1
2013-12-11 23:45:07 +00:00
# Revision 33 some test cases from CLDR
array( 0, 'i = 1 and v = 0 or i = 0 and t = 1', '0.1', 'pt one' ),
array( 0, 'i = 1 and v = 0 or i = 0 and t = 1', '0.01', 'pt one' ),
array( 0, 'i = 1 and v = 0 or i = 0 and t = 1', '0.10', 'pt one' ),
array( 0, 'i = 1 and v = 0 or i = 0 and t = 1', '0.010', 'pt one' ),
array( 0, 'i = 1 and v = 0 or i = 0 and t = 1', '0.100', 'pt one' ),
array( 1, 'i = 1 and v = 0 or i = 0 and t = 1', '0.0', 'pt other' ),
array( 1, 'i = 1 and v = 0 or i = 0 and t = 1', '0.2', 'pt other' ),
array( 1, 'i = 1 and v = 0 or i = 0 and t = 1', '10.0', 'pt other' ),
array( 1, 'i = 1 and v = 0 or i = 0 and t = 1', '100.0', 'pt other' ),
Pass phpcs-strict on some test files (11/11) Woo! Change-Id: I9fc116dfdf18c2772d047adb5bb14535d0bd39ed
2014-04-24 17:52:51 +00:00
// @codingStandardsIgnoreStart Ignore Generic.Files.LineLength.TooLong
Plural rules: updates for UTS #35 Rev 33 * New operands i, v, w, f, t * New operators =, !=, % * Ignore "samples", which are basically unit tests embedded in rule specifications * Ignore the new "other" rules, which have an empty condition. It doesn't really makes sense to parse them, since the empty condition means special handling should be done in the caller, it is not equivalent to an unconditional true or false. * Trailing zero support requires that the input number be a string. Documented this. * Fixed some comments * Added test cases for new features Bug: 56931 Change-Id: I96986c0c664f785e75b0a4ced2ec9e37b72681c1
2013-12-11 23:45:07 +00:00
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '2', 'bs few' ),
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '4', 'bs few' ),
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '22', 'bs few' ),
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '102', 'bs few' ),
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '0.2', 'bs few' ),
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '0.4', 'bs few' ),
array( 0, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '10.2', 'bs few' ),
array( 1, 'v = 0 and i % 10 = 2..4 and i % 100 != 12..14 or f % 10 = 2..4 and f % 100 != 12..14', '10.0', 'bs other' ),
Pass phpcs-strict on some test files (11/11) Woo! Change-Id: I9fc116dfdf18c2772d047adb5bb14535d0bd39ed
2014-04-24 17:52:51 +00:00
// @codingStandardsIgnoreEnd
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
);
return $tests;
}
function invalidTestCases() {
$tests = array(
array( 'n mod mod 5 is 1', 'mod mod' ),
array( 'n', 'just n' ),
array( 'n is in 5', 'is in' ),
);
Update code formatting Change-Id: I16a9b42651f1cfb1a70dffbb67b7b83dfeb90d03
2013-04-26 12:00:22 +00:00
CLDR plural parser in PHP Wrote a CLDR plural rule parser to replace the eval()-based one from I58a9cdfe. It converts the infix notation of the XML files to a sanitized RPN notation, referred to in external interfaces as the "compiled" form. The RPN notation is cached and then executed by a fast non-validating evaluator. Timings for the largest rule in the XML file are ~1.2ms for compilation and ~200us for execution. Also: * Lazy-load the plural rules when recache() requests them, instead of loading them for every request. * Language::convertPlural() needs integer keys, and CLDR only gives string keys. The previous code was not mapping them so it didn't work at all. I just mapped them in the order they appear in the XML file, i.e. the first rule becomes MediaWiki's $pluralForm=0, the second becomes $pluralForm=1, etc. Not sure if there is a more rigorous way to do it. Change-Id: I65ee788c1a8e5ee2ede2091990d86eb722749dd3
2012-08-16 10:36:08 +00:00
return $tests;
}
}
Reference in a new issue Copy permalink