If a client submits data that is not NFC-normalized Unicode or that
contains C0 controls other than HT, LF, and CR, it gets normalized before
the API ever sees it. Which can lead to difficult-to-handle bugs when,
for example, a title is subject to normalization so the client can't
find the specific title it submitted anywhere in the response (T139130).
This patch does two things:
* Detects when normalization was applied to an input value (at the
MediaWiki level, anyway; if PHP or earlier does it we're just out of
luck) and add a warning to that effect.
* For ApiPageSet's 'titles' parameter, split into the individual titles
and add them to the 'normalized' list in the response. This requires
encoding the pre-normalized strings to avoid ApiResult's own
normalization.
Bug: T29849
Bug: T144071
Change-Id: I215fd3edd7a5e1b45292e60768bf6dd5ad7f34de
When a multi-valued parameter's value begins with U+001F, the values
will be split on that character instead of pipes. This will be useful
for things such as action=options&change= or meta=allmessages&amargs=.
Since MediaWiki doesn't otherwise accept C0 control characters
(WebRequest::getVal() replaces them with �), there's no possibility that
this will conflict with a literal use of U+001F.
Special:ApiSandbox and mw.Api are updated to make use of this, with the
latter having an option to disable the behavior in case something is
depending on [ 'foo', 'bar|baz' ] turning into 'foo|bar|baz'.
Pipe is still used as the separator when the value doesn't begin with
U+001F, and will be forever since it's generally more human-friendly and
is needed for backwards compatibility with basically every API client in
existence. The requirement that the value begin with U+001F, rather than
simply contain U+001F, is to avoid clients having to somehow
special-case "param=foo|bar" where that's intended to be a single value
"foo|bar" rather than two values "foo" and "bar".
Bug: T141960
Change-Id: I45f69997667b48887a2b67e93906364a652ace5a
