Implicitly marking parameter $... as nullable is deprecated in php8.4,
the explicit nullable type must be used instead
Created with autofix from Ide15839e98a6229c22584d1c1c88c690982e1d7a
Break one long line in SpecialPage.php
Bug: T376276
Change-Id: I807257b2ba1ab2744ab74d9572c9c3d3ac2a968e
This adds support for serializing/deserializing objects which
implement the JsonCodecable interface from the wikimedia/json-codec
library used by Parsoid. JsonCodecable allows customizing the encoding
of objects of a given class using a class-specific codec object, and
JsonCodecable is an interface which is defined and can be used outside
mediawiki core.
In addition json-codec supports deserialization in the presence of
aliased class names, fixing T353883.
Backward and forward compatibility established via the mechanism
described in
https://www.mediawiki.org/wiki/Manual:Parser_cache/Serialization_compatibility
Test data generated by this patch was added in
I109640b510cef9b3b870a8c188f3b4f086d75d06 to ensure forward
compatibility with the output after this patch is merged.
Benchmarks:
PHP 7.4.33 PHP 8.2.19 PHP 8.3.6
BEFORE AFTER BEFORE AFTER BEFORE AFTER
Serialize: 926.7/s 1424.8/s 978.5/s 1542.4/s 1023.5/s 1488.6/s
Serialize (assoc): 930.2/s 1378.6/s 974.6/s 1541.9/s 1022.4/s 1463.4/s
Deserialize: 1942.7/s 1961.3/s 2118.8/s 2175.9/s 2129.8/s 2063.5/s
Deserialize (assoc): 1952.0/s 1905.7/s 2107.5/s 2192.1/s 2153.3/s 2011.1/s
These numbers definitely do not have as many significant digits as
written here. But they should be sufficient to demonstrate that
performance is not impaired by this patch and in fact serialization
speed improves slightly.
Bug: T273540
Bug: T327439
Bug: T346829
Bug: T353883
Depends-On: If1d70ba18712839615c1f4fea236843ffebc8645
Change-Id: Ia1017dcef462f3ac1ff5112106f7df81f5cc384f
get_debug_type() does the same thing but better (spelling type names
in the same way as in type declarations, and including names of
object classes and resource types). It was added in PHP 8, but the
symfony/polyfill-php80 package provides it while we still support 7.4.
Also remove uses of get_class() and get_resource_type() where the new
method already provides the same information.
For reference:
https://www.php.net/manual/en/function.get-debug-type.phphttps://www.php.net/manual/en/function.gettype.php
In this commit I'm only changing code where it looks like the result
is used only for some king of debug, log, or test output. This
probably won't break anything important, but I'm not sure whether
anything might depend on the exact values.
Change-Id: I7c1f0a8f669228643e86f8e511c0e26a2edb2948
get_debug_type() does the same thing but better (spelling type names
in the same way as in type declarations, and including names of
object classes and resource types). It was added in PHP 8, but the
symfony/polyfill-php80 package provides it while we still support 7.4.
Also remove uses of get_class() and get_resource_type() where the new
method already provides the same information.
For reference:
https://www.php.net/manual/en/function.get-debug-type.phphttps://www.php.net/manual/en/function.gettype.php
To keep this safe and simple to review, I'm only changing cases where
the type is immediately used in an exception message.
Change-Id: I325efcddcb58be63b1592b9c20ac0845393c15e2
This is to make it clearer that they're related to converting serialized
content back into JSON, rather than stating that things are not
representable in JSON.
Change-Id: Ic440ac2d05b5ac238a1c0e4821d3f2d858bc3d76
This patch introduces a namespace declaration for the
MediaWiki\Json to FormatJson and establishes a class
alias marked as deprecated since version 1.43.
Bug: T353458
Change-Id: I5e1311e4eb7a878a7db319b725ae262f40671c32
Also fixes JsonCodeTest::testInvalidJsonData() which was misusing the
data provided by ::provideSimpleTypes().
Change-Id: Ia654359e0fdec3ad546e8bea2e9133c142f0f144
This class is used heavily basically everywhere, moving it to Utils
wouldn't make much sense. Also with this change, we can move
StatusValue to MediaWiki\Status as well.
Bug: T321882
Depends-On: I5f89ecf27ce1471a74f31c6018806461781213c3
Change-Id: I04c1dcf5129df437589149f0f3e284974d7c98fa
A valid JSON serialization is an instance of PHP's stdClass. A check
with is_object() is not sufficient in this case because it includes
everything else that's also a class in PHP.
This should help to uncover programming errors like the one in
I969d8c4.
Bug: T312589
Change-Id: I917d49944497b19909a9a1d1e2861e86e7a0aca8
Add a type annotation when encoding `stdClass` objects so that we can
be sure to decode them as objects instead of arrays.
This avoids issues such as that seen in the Graph extension (T312589)
where an extension data key is stored as a stdClass. If ParserOutput
was computed fresh, a subsequent getExtensionData(..) call will return
a stdClass object, but if the ParserOutput was cached, getExtensionData()
would return an array. After this change the return type is always
consistent.
Properly handle nested objects: encode all object values returned by
JsonSerializable::jsonSerialize() (so that client is not responsible
for implementing this correctly), and decode all object values *before*
calling JsonUnserializable::newFromJsonArray (again, so that the
client is not responsible for decoding its property values). The new
behavior matches how serialize/unserialize is handled in the 'naive'
JsonUnserializable{Sub,Super}Class test cases; ParserOutput (the only
users of JsonCodec in core) was doing an extra manual decode for
the ExtensionData array in ParserOutput::initFromJson that is no longer
necessary.
The GrowthExperiments and SemanticMediaWiki extensions were working
around the non-recursive nature of JsonCodec; this patch depends on
patches to GrowthExperiments to make it agnostic about whether object
unserialization occurs before or after ::newFromJsonArray() is called,
which can then be further cleaned up once this is released.
A pull request for SemanticMediaWiki has also been submitted.
Bug: T312589
Depends-On: I3413609251f056893d3921df23698aeed40754ed
Change-Id: Id7d0695af40b9801b42a9b82f41e46118da288dc
These two interfaces' methods have tentative return types in PHP 8.1,
which causes code without the type hints to raise warnings. Where the
type hint is "mixed", we need to use the special declaration
[\ReturnTypeWillChange] in a comment to suppress the warning as long as
we still support PHP < 8.0, which doesn't have a "mixed" type hint.
Bug: T289879
Change-Id: I1a126e602e92b8d13c7795eb6d790effd5ddc986
- Removed the str_replace() call to replace unescaped line terminators
if UTF8_OK is set. PHP 7.1 and later escape these by default.
The speedup isn't much at all (about 1% in my testing when encoding an
API siteinfo result taken from enwiki). Perhaps it's not surprising
given the way str_replace() works[1]. Still, it's better not to spend
CPU time looking for characters that will not occur.
- Changed the algorithm for the optional spaces-to-tabs conversion when
pretty printing. Instead of replacing one indent level throughout the
entire string before replacing the next level, use a regex to replace
in one pass. This is usually faster now that PHP 7 enables PCRE's JIT
compiler by default. Without JIT, the regex was often slower.
The speedup can be large for deeply nested data. For example, in my
testing the languages/i18n data took about 8% less time to encode as
tab-indented JSON, yet the API site info result took about 45% less.
(This, of course, isn't actually relevant to the API even when pretty
printed output is requested, because ApiFormatJson uses the default
indent string of four spaces, which will always be faster unless
support for tab indentation is added to PHP's json extension.)
- Set options using if statements instead of the ternary operator. This
is the clearer way, and maybe the slightly faster one, skipping the
assignment when the flags do not need to be set.
[1]: https://github.com/php/php-src/blob/PHP-8.0.10/ext/standard/string.c#L2969
Change-Id: Iebb1df0264e335a1819956710eeacf6d6b8f1471
The comment added in b9461e3f1c is incorrect. This is actually a
decode error, so is relevant to FormatJson::parse().
Change-Id: I3cc33f0f260c0ba4fe96fb75565f52d089b9a975
DEPLOY: Set $wgOldRevisionParserCacheExpireTime = 0 in production first!
Bug: T267832
Depends-On: I3c73f5d9f6a54e2736600e8f9506659a3fb0e7f6
Change-Id: I0fe275b4991f1bf89c7bb587132bc4fb0ea862e2
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.
Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
Three of the errors are encode errors that won't be emitted when we're
trying to decode JSON, so we can ignore those lines of code.
JSON_ERROR_UTF16 is a new error code in PHP 7.0, so add that in.
Improve test coverage while we're at it. The UTF16 test case was
copied from php-src/ext/json/tests/bug62010.phpt.
Change-Id: I79aa0db3d967d512611f8521bb052af36c3cda8e
PHP JSON decoding has surprising behavior on some edge cases.
Documented this via comments, added related tests, and tweaked
related CommentStore code.
Bug: T206411
Change-Id: I6927fdaf616b37a04d81a638a0ed257afac9b844
"continue" statements are equivalent to "break". In PHP 7.3, will generate a warning.
Bug: T200595
Change-Id: I244ecb2e1ce5a76295f014fb1becd8d263196846
* Use PHP 5.6 constant expression support in definition of ALL_OK.
* Remove one level of nesting in encode(). Follows up I801eaffc.
* Update HTML5 section number in doc comment for XMLMETA_OK.
* Made other minor doc comment fixes, such as capitalizing "JSON".
* Not done: changing $badChars and $badCharsEscaped to constants.
This will have to wait until HHVM 3.18 support is dropped.
Change-Id: I06413dfe0fedddfd20d3e375eadd9daad6d6230e
In cases where we're operating on text data (and not binary data),
use e.g. "\u{00A0}" to refer directly to the Unicode character
'NO-BREAK SPACE' instead of "\xc2\xa0" to specify the bytes C2h A0h
(which correspond to the UTF-8 encoding of that character). This
makes it easier to look up those mysterious sequences, as not all
are as recognizable as the no-break space.
This is not enforced by PHP, but I think we should write those in
uppercase and zero-padded to at least four characters, like the
Unicode standard does.
Note that not all "\xNN" escapes can be automatically replaced:
* We can't use Unicode escapes for binary data that is not UTF-8
(e.g. in code converting from legacy encodings or testing the
handling of invalid UTF-8 byte sequences).
* '\xNN' escapes in regular expressions in single-quoted strings
are actually handled by PCRE and have to be dealt with carefully
(those regexps should probably be changed to use the /u modifier).
* "\xNN" referring to ASCII characters ("\x7F" and lower) should
probably be left as-is.
The replacements in this commit were done semi-manually by piping
the existing "\xNN" escapes through the following terrible Ruby
script I devised:
chars = eval('"' + ARGV[0] + '"').force_encoding('utf-8')
puts chars.split('').map{|char|
'\\u{' + char.ord.to_s(16).upcase.rjust(4, '0') + '}'
}.join('')
Change-Id: Idc3dee3a7fb5ebfaef395754d8859b18f1f8769a
The PHP bug 66021 <https://bugs.php.net/bug.php?id=66021> was fixed by
https://github.com/php/php-src/pull/518 and is included in PHP 5.4.28+
and PHP 5.5.12+.
This workaround is not necessary anymore because the minimum PHP
version for MediaWiki is 7.0.0+.
Change-Id: I801eaffc253fd88e0d3c87cfe97777837bd3902d
Searched for /([^\d\w\s\)\]]\s*)- \d/ to find potential issues.
It seems there's no PHPCS check for this, huh.
Also fixed typo in a comment in LoginSignupSpecialPage.
Change-Id: Iaab1a1f5a9f234971e550e7909aa5c3e0c02a983
This fixes the outstanding mis-spaced cast operators to bring them
into line with the coding standards on mediawiki.org (and with the
more common usage within this codebase).
Bug: T149545
Change-Id: Ib7bcf95bbee83d20c05f6d621ce7b4e1fb58a347
MediaWiki now only works with PHP versions that are new enough
to have the encoding options required by encode54(). So fold
that into encode() and remove encode53() and prettyPrint().
Change-Id: I6b22daf8fa01ef608efbde9c6aecdbb5ce03e2b9
This makes sure static analyzers don't warn for supposedly unsafe
code accessing variables as strings when they could be boolean after
having only checked against false.
https://github.com/scrutinizer-ci/php-analyzer/issues/605
Change-Id: Idb676de7587f1eccb46c12de0131bea4489a0785
Add stripComments method that can be used to remove single line and
multiline comments from an otherwise valid JSON string. Inspired by the
comment removal code in redisJobRunnerService and discussions on irc
about the Extension registration RFC.
Change-Id: Ie743957bfbb7b1fca8cb78ad48c1efd953362fde
Constant values were changed to be above 0xFF - this way
we can easily decide to allow depth-parsing-limit to be OR-able:
FormatJson::parse( $value, 30 | FormatJson::FORCE_ASSOC )
Follows-up Ic0eb0a7 and I1c4f37a.
Change-Id: I9bfd67a5ca4ea1d399821549c7e63ffdecd56ad1
Removes trailing commas from json text when parsing
Solves very common cases like [1,2,3,]
Resulting status will be set to OK but not Good to warn caller
Change-Id: Ic0eb0a711da3ae578d6bb58d7474279d6845a4a7
* Returns Status object that will contain decoded value on success
* Adds i18n messages for all available PHP JSON errors
ATTN Translation team: please copy these messages:
gwtoolset-json-error-depth => json-error-depth
gwtoolset-json-error-state-mismatch => json-error-state-mismatch
gwtoolset-json-error-ctrl-char => json-error-ctrl-char
gwtoolset-json-error-syntax => json-error-syntax
gwtoolset-json-error-utf8 => json-error-utf8
Change-Id: I1c4f37aaabad369b75a1fbd223fad27ebcfe1c3c
Follows-up bec7e8287c. The comment "Can be removed once we require
PHP >= 5.4.28, 5.5.12, 5.6.0" relies on some assumptions that might
later prove to be incorrect:
* That the fix won't be reverted from any of those PHP versions
(e.g. if deemed to break BC)
* That the bug will be fixed in PECL jsonc and jsond, as well as in
HHVM
* That we don't need to support older versions of those once we
require one of the mentioned PHP versions
Change-Id: I67034c561d54d37dee961ada8c9cf5ccfd113da1
The patch[1] for PHP bug 66021[2], which removes the same undesirable
whitespace that WS_CLEANUP_REGEX does, has been merged into php-src.
Subsequent PHP versions having the patch shouldn't have to take the
10-20% performance hit from that workaround.
[1]: https://github.com/php/php-src/commit/82a4f1a1a287
[2]: https://bugs.php.net/bug.php?id=66021
Change-Id: I717a0e164952cc6ace104f13f6236e86c4ab8b58
