Change I5521b7652f (commit 682a19e9f6) increased the number by roughly
one order of magnitude, which leaves me nervous that the tests might
start failing again if enough other tests are run in the same wiki to
let the increased revision IDs exist. Let’s bump it further to make that
much less likely (though not impossible).
1219844647 is an arbitrary large number; my intention in choosing a
random-looking number (rather than, say, 1234567890) is that it’s easier
to search for, both in codesearch and in commit messages, if it should
ever pop up in an error message.
Bug: T366142
Change-Id: I8186d9d46bc2a3f5ec04b38aab5cbe85e609835a
Follows-Up: I5521b7652faca9821fa08a9987a9452a4c555203
The default used to be 'ucs2' when linting but
Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80 stopped setting the offsetType
in the environment options, which changed the output to Parsoid's
default, 'byte'.
For the parsing paths, pagebundles are now post-processed and
non-pagebundles were left as a TODO.
However, for the lint paths, it was always ok to continue setting the
environment option because Parsoid is called directly and doesn't go
through the ParserOutputAccess. The above patch was trying to limit
Parsoid specific options to that.
Bug: T365284
Change-Id: I8389c2f53b399b39a9f1d908a38aecb3abcb15ef
These tests fail for wikis with many revisions, and even in CI when
running after a suite that generates many edits such as Wikibase.
Bug: T366142
Change-Id: I5521b7652faca9821fa08a9987a9452a4c555203
Mostly this has a bunch of whitespace changes from the
template-curly-spacing and brace-style rules being set
to align with other spacing rules.
Change-Id: I4609c52a4ef426ad1f35fb4bfe447bb08323a8e8
Request that only specify a title, no revision, should get the page
content directly.
Bug: T350359
Change-Id: Ia461cce0df63c05c6f8f94275f6b94f81323d171
This was missed when fixing a similar issue for core endpoints.
I also deleted some dead code in ParsoidHandler, instead of fixing
it to also generate relative redirects.
Bug: T350219
Bug: T349001
Change-Id: If3fe901723dcae9a22806650a21a79778354d8c5
* This is in service of a followup patch that merges ParsoidOutputAccess
and ParserOutputAccess. We want to eliminate all Parsoid-specific options
that aren't part of ParserOptions and aren't easily supportable via
html2html transforms.
* offsetType conversion relies on Parsoid code that is a bit entangled
with env, siteconfig (and extension configs), page source, etc. It
could all be refactored but once the html2html output transformation
framework lands, we could potentially use that to call Parsoid to do
these transforms by exposing such transforms to the framework.
* In this patch, outputContentVersion that isn't the default major HTML
version is no longer support. It could potentially be supported via the
downgrade functionality in Parsoid in the future, or we might decide
to re-enable multiple outputContentVersion selection in the future
if such a use case arises. But, there are no plans to bump the major
HTML version in the near future while we work on read views.
* Rather than delete associated tests, I've marked them skipped so that
they can re-enabled when this support is added back.
Bug: T347426
Change-Id: Ibede4acd68e944512f6d00763d29c6b1605d67eb
It is very easy for developers and maintainers to mix up "internal
MediaWiki language codes" and "BCP-47 language codes"; the latter are
standards-compliant and used in web protocols like HTTP, HTML, and
SVG; but much of WMF production is very dependent on historical codes
used by MediaWiki which in some cases predate the IANA standardized
name for the language in question.
Phan and other static checking tools aren't much help distinguishing
BCP-47 from internal codes when both are represented with the PHP
string type, so the wikimedia/bcp-47-code package introduced a very
lightweight wrapper type in order to uniquely identify BCP-47 codes.
Language implements Bcp47Code, and LanguageFactory::getLanguage() is
an easy way to convert (or downcast) between Bcp47Code and Language
objects.
This patch updates the Parsoid integration code and the associated
REST handlers to use Bcp47Code in APIs so that the standalone Parsoid
library does not need to know anything about MediaWiki-internal codes.
The principle has been, first, to try to convert a string to a
Bcp47Code as soon as possible and as close to the original input as
possible, so it is easy to see *why* a given string is a BCP-47 code
(usually, because it is coming from HTTP/HTML/etc) and we're not stuck
deep inside some method trying to figure out where a string we're
given is coming from and therefore what sort of string code it might
be. Second, we've added explicit compatibility code to accept
MediaWiki internal codes and convert them to Bcp47Code for backward
compatibility with existing clients, using the @internal
LanguageCode::normalizeNonstandardCodeAndWarn() method. The intention
is to gradually remove these backward compatibility thunks and replace
them with HTTP 400 errors or wfDeprecated messages in order to
identify and repair callers who are incorrectly using
non-standard-compliant language codes in web standards
(HTTP/HTML/SVG/etc).
Finally, maintaining a code as a Bcp47Code and not immediately
converting to Language helps us delay or even avoid full loading of a
Language object in some cases, which is another reason to occasionally
push Bcp47Code (instead of Language) down the call stack.
Bug: T327379
Depends-On: I830867d58f8962d6a57be16ce3735e8384f9ac1c
Change-Id: I982e0df706a633b05dcc02b5220b737c19adc401
Allow clients to use an If-Match header with the
transform/html/to/wikitext endpoint.
Bug: T310464
Needed-By: Ifb1c40a0044f04fb339b00630fbca9190a1bce51
Change-Id: Ida81a314f015e205f2081c68a82d486145097c92
This restores change Ie430acd0753880d88370bb9f22bb40a0f9ded917.
This reverts commit ab6baad1a5.
NOTE: Also needs the patch the fixes the original reason for the
revert: Ief721c23ed9a57d781cfdac625a62113f22f87a5
Change-Id: Ic48db1b5fdff1dfd4f2d2643d64252e5fc721e79
NOTE: This causes Parsoid output to be written to the parser cache.
This should be unconditional in the future, but for now it is
controled by wgTemporaryParsoidHandlerParserCacheWriteRatio.
This change affects the following endpoints that use the wt2html method:
* /coredev/v0/transform/wikitext/to/html in core
* /{domain}/v3/transform/wikitext/to/html from parsoid
* /{domain}/v3/page/html/{title} from parsoid
The /v1/page/{title}/html endpoint is not affected, since it
doesn't use wt2html, but has always been using HtmlOutputRendererHelper
directly.
Bug: T322672
Depends-On: Ic37f606bb51504c8164d005af55ca9a65f595041
Change-Id: Ie430acd0753880d88370bb9f22bb40a0f9ded917
* Apply Legacy Temporary redirects (302) if page is a redirect in order
to have feature parity with RESTBase
* Check for normalization redirects and execute permanent redirects (301)
* Add/Update mocha tests for the redirects functionality
* Add query parameter 'redirect=no' check to bypass redirect logic
* Unit tests to check status code and location headers
Bug: T301372
Change-Id: I841c21d54a58e118617aaf5e2c604ea22914adaa
The tests for size limits did not catch an issue introduced by
If09afc4b933e, which caused resource limits to trigger early, since they
were now being compared to the size in bytes, rather than characters.
The reason the tests didn't protect us is threefold:
- They only check the error returned when the resource sizes is one over the limit.
They don't check that the error is NOT returned when the size is one under the limit.
- They did not test with a multi-byte character.
- They were disabled, because the limits are quite high, and the e2e test can not change them.
This patch is an attempt to fix all three issues.
Depends-On: I40901a1204b3c698895a836bf3b605239878d1fe
Change-Id: I2aead24cb7f47eb1267fdd2954a7c7e45dd4ed51
If we don't have a render id given, but we do have a revision id, we can
fall back to the current rendering that revision to provide a baseline for
selser. This is better than no selser. On wikis that do not heavily rely
on templates, or where templates rarely change while an edit is in
progress, this will produce a clean diff.
Bug: T318398
Change-Id: If7612cc6e64d1f1243289b7d6ba96c71f09fe15d
Parsoid needs the original rendering in order to apply
selective serialization (selser). The page/{title}/html endpoint
can stash the rendering, and now the transform endpoint can make use
of the stashed rendering.
Bug: T310464
Change-Id: Ia58043ed3aa1eb12731d82aa87606c82ec63f663
Parsoid supports other source formats besides wikitext.
This patch improves support for non-wikitext content by removing
assumptions about the source type.
Change-Id: I5480ff200a93026cea7f1542e12834b06ac6f730
This renames TransformContext to HTMLTransformInput. It is becoming a
wrapper around the input HTML, with a bunch of optional context data
attached.
This introduces a factory method for HTMLTransformInput, so we can
extract knowledge about the structure of the $attribs array from
HTMLTransformInput.
This also allows us to inject Document objects and perhaps PageBundle
objects, instead of just arrays.
Change-Id: I66f9c5dbb50c6bf1f582adad7766422216482402
Move HTML and other complex input and output data from giant strings into files.
This makes them easier to read, modify, and compare.
Change-Id: Iafe2638e4eda903e1064b05adaa80a39ff5028f9
Page bundles should not be part of the publicly accessible
API and if we really need it, we can expose it and consume
internally. Hence, remove support for this transform from the
endpoint.
This patch falls back to the behavior before for backwards
compatibility before we fully switch over to using the transforms
in core. We still need to support both core & restbase for now.
NOTE: API integration tests have been updated to status code
404 since we're removing support for it now. If we need it,
we'll expose via a different endpoint and introduce tests.
Bug: T311477
Change-Id: I5577f61f7ae7da2a4d3a78d9ce962997466d550c
This changes the Transform.js e2e test to not break when the parsoid
schema version is getting higher. We are only guaranteeing stability
according to the semver caret semantics to clients, so that is what
the e2e test should assert.
This allows tests to pass when raising the version of the parsoid lib in
composer.json. This also allows core tests to pass when run against the
development branch of the parsoid repo.
Change-Id: I6d7db6a05c48de8a57f83e4c8af38ab50271297a
RESTBase used this to check if the domain sent by the client
matches the domain in the restbase config under the hood. Since
this code is moving to core, this assertion is no longer needed.
Bug: T301370
Change-Id: I01a4f35b81c31d106671e5c829d317a41687fd7a
