Tests should create fixtures if and when they need them. Create test
pages explicitly in tests that were expecting them to exist.
Bug: T342428
Change-Id: I552420bb857388cb0873f7afc4e8b15b88388937
* Illegal string offset and invalid argument supplied to foreach, due to incorrect type information
* Array internal pointer reset is unnecessary
* $hookData unused since MW 1.35 due to incomplete revert
* array_push() with single element
* Unnecessary sprintf()
* for loop can be replaced with str_repeat()
* preg_replace() can be replaced with rtrim()
* array_values() call is redundant
* Unnecessary cast to string
* Unnecessary ternary. Often the result relies on short-circuit evaluation, but I find it more readable nonetheless.
Change-Id: I4c45bdb59b51b243fa96286bec8b58deb097d707
Just methods where adding "static" to the declaration was enough, I
didn't do anything with providers that used $this.
Initially by search and replace. There were many mistakes which I
found mostly by running the PHPStorm inspection which searches for
$this usage in a static method. Later I used the PHPStorm "make static"
action which avoids the more obvious mistakes.
Bug: T332865
Change-Id: I47ed6692945607dfa5c139d42edbd934fa4f3a36
The page/html endpoint should only return an ETag if stashing was
requested. Otherwise, the ETag is meaningless.
Bug: T331629
Change-Id: I55d7c5e33ef7275695ee93e2937da1a998e2eda3
Allow clients to use an If-Match header with the
transform/html/to/wikitext endpoint.
This follows up on Ida81a314f015e205f2081c68a82d486145097c92
(reverted and reapplied)
It adds support for stashing in wt2html, enabling it for Parsoid's
page/html endpoint. It also ensures we are only emitting ETags if
stashing is enabled.
This also removes handling for use-stash from ParsoidHandler,
which did nothing.
Bug: T310464
Bug: T331629
Needed-By: I08f1388faaccef6c1d9a393f8011011d30a25ec7
Change-Id: I9d6eaf45d5b4978afc17493720777e77f0e645b2
It is very easy for developers and maintainers to mix up "internal
MediaWiki language codes" and "BCP-47 language codes"; the latter are
standards-compliant and used in web protocols like HTTP, HTML, and
SVG; but much of WMF production is very dependent on historical codes
used by MediaWiki which in some cases predate the IANA standardized
name for the language in question.
Phan and other static checking tools aren't much help distinguishing
BCP-47 from internal codes when both are represented with the PHP
string type, so the wikimedia/bcp-47-code package introduced a very
lightweight wrapper type in order to uniquely identify BCP-47 codes.
Language implements Bcp47Code, and LanguageFactory::getLanguage() is
an easy way to convert (or downcast) between Bcp47Code and Language
objects.
This patch updates the Parsoid integration code and the associated
REST handlers to use Bcp47Code in APIs so that the standalone Parsoid
library does not need to know anything about MediaWiki-internal codes.
The principle has been, first, to try to convert a string to a
Bcp47Code as soon as possible and as close to the original input as
possible, so it is easy to see *why* a given string is a BCP-47 code
(usually, because it is coming from HTTP/HTML/etc) and we're not stuck
deep inside some method trying to figure out where a string we're
given is coming from and therefore what sort of string code it might
be. Second, we've added explicit compatibility code to accept
MediaWiki internal codes and convert them to Bcp47Code for backward
compatibility with existing clients, using the @internal
LanguageCode::normalizeNonstandardCodeAndWarn() method. The intention
is to gradually remove these backward compatibility thunks and replace
them with HTTP 400 errors or wfDeprecated messages in order to
identify and repair callers who are incorrectly using
non-standard-compliant language codes in web standards
(HTTP/HTML/SVG/etc).
Finally, maintaining a code as a Bcp47Code and not immediately
converting to Language helps us delay or even avoid full loading of a
Language object in some cases, which is another reason to occasionally
push Bcp47Code (instead of Language) down the call stack.
Bug: T327379
Depends-On: I830867d58f8962d6a57be16ce3735e8384f9ac1c
Change-Id: I982e0df706a633b05dcc02b5220b737c19adc401
The current signature of the various execute methods only takes a
boolean parameter to determine if the session should be safe against
CSRF, but that does not give callers fine-grained control over the
Session object, including setting a specific token.
Also, do not use createNoOpMock in getSession(), since it implies
strong assertions on what methods are called. This way, getSession
can also be used to get a simple mock session that tests may further
manipulate.
Make $csrfSafe parameter of SessionHelperTestTrait::getSession
mandatory. This way, callers are forced to think what makes sense in
each use case. The various methods in HandlerTestTrait now default to
a session that is safe against CSRF. This assumes that most REST
handlers don't care about the session, and that any handler that does
care about the session and where someone needs to test the behaviour
in case of bad/missing token will explicitly provide a Session that
is NOT safe against CSRF.
Typehint the return value of Session(Backend)::getUser so that PHPUnit
will automatically make it return a mock User object even if the method
is not explicitly mocked. Remove a useless PHPUnit assertion -- setting
the return value to be X and then veryfing that is equal to X is a
tautology, and can only fail if the test itself is flawed (as was the
case, since it was using stdClass as the return type for all
methods). Remove the getUser test case altogether, there's no way to
make it work given the DummySessionBackend, and the test isn't that
helpful anyway. More and more methods will have the same issue as soon
as their return value is typehinted.
Follow-up: I2a9215bf909b83564247ded95ecdb4ead0615150
Change-Id: Ic51dc3e7bf47c81f2ac4705308bb9ecd8275bbaf
Mixing Handlers with Helpers doesn't look nice for consistency
reasons. Helpers should be in their own place (grouped) in the
Handlers directory as they're really "helpers for the handlers".
Change-Id: Ieeb7a0a706a4cb38778f312bfbfe781a1f366d14
This introduces an interface HtmlOutputHelper that is implemented
by both HtmlMessageOutputHelper or HtmlOutputRendererHelper based
on the page we're dealing with.
Bug: T323558
Change-Id: I1fb8dcc5cc05ce3f32f3c1862b88045f1c8e612b
When parsoid was dealing with content for content models that it does
not support, the corresponding pagebundle used `null` as the version
which will break when converting the it to JSON.
This patch fixes this by just using a version which is not specified
like 0.0.0
A better fix would be for page bundle to not explode when the version is
set to null. The other real fix is for Parsoid to not ask for rendering of
pages that are not wikitext.
Bug: T325137
Change-Id: Iff1ce432d1b2d30f3f74c53a0602c11034db5874
Loosening the version checks in the integration tests is akin to what
was done for api-testing in I6d7db6a05c48de8a57f83e4c8af38ab50271297a
and I317ce587e62f9e94bbafbdabac64156237c4f1e3.
The tightly coupled tests were added in Ieb4b41375d521893f95e2fcb5f4984e7b5a2364c
The change to the condition when original html needs downgrading was
exposed by the test added in Ic3cc3a598f32cad6122964cb8a7376a56be9129f
though not exercised because edited and re-parsed html had the same
versions. Without it, in-flight edits on wikis that might request a
re-parse (think private wikis) when a new semantically equivalent
version is deployed could fails due to trying to find a downgrade path
and none being there.
Needed-By: Iabab0c093dcb21e28c643be6e85cf1a7b54cd999
Change-Id: I33e70df750c6d4b082281fdc8bacdea72662832a
Fixes the reason for reverting Ie430acd0753880d88370bb9f22bb40a0f9ded917:
The issue was that with my patch, the transform/wikitext/to/html started
ignoring the offsetType field in the body. So the offsetType used in the
response (or stashed data) would always be 'byte'.
But the roundtrip-test.js scripts requests 'ucs2'.
This causes an error when sending the HTML and data-parsoid back to
transform/html/to/wikitext, again with offsetType:'ucs2': the offsetType
embededed in data-parsoid will be byte, and the mismatch causes a 400
to be returned. This broke the roundtrip-test.js script.
The fix is to no ignore the offsetType specified in the request body.
Change-Id: Ief721c23ed9a57d781cfdac625a62113f22f87a5
This restores change Ie430acd0753880d88370bb9f22bb40a0f9ded917.
This reverts commit ab6baad1a5.
NOTE: Also needs the patch the fixes the original reason for the
revert: Ief721c23ed9a57d781cfdac625a62113f22f87a5
Change-Id: Ic48db1b5fdff1dfd4f2d2643d64252e5fc721e79
NOTE: This causes Parsoid output to be written to the parser cache.
This should be unconditional in the future, but for now it is
controled by wgTemporaryParsoidHandlerParserCacheWriteRatio.
This change affects the following endpoints that use the wt2html method:
* /coredev/v0/transform/wikitext/to/html in core
* /{domain}/v3/transform/wikitext/to/html from parsoid
* /{domain}/v3/page/html/{title} from parsoid
The /v1/page/{title}/html endpoint is not affected, since it
doesn't use wt2html, but has always been using HtmlOutputRendererHelper
directly.
Bug: T322672
Depends-On: Ic37f606bb51504c8164d005af55ca9a65f595041
Change-Id: Ie430acd0753880d88370bb9f22bb40a0f9ded917
Otherwise the mocked getLangCode() method returns null, which is not
allowed any more in PHP 8.1.
Bug: T289926
Required-For: I7e026cca216aba24ee5d5662b6fca322b3cec9ae
Change-Id: I178def7f03a44f6b49cdb461d9ab340e1c89517f
When no original HTML and no etag is provided,
selser should still be attempted based on a rendered
version of the old wikitext.
Change-Id: Ic3cc3a598f32cad6122964cb8a7376a56be9129f
* We will have several kinds of HTML transformations.
Rename HTMLTransform to indicate that its for converting HTML to Content
objects.
* Using Naming Convention 'Html' instead of 'HTML'
Change-Id: I506f3303ae8f9e4db17299211366bef1558f142c
ParsoidHandler should pass the metrics object from the
SiteConfig to HtmlInputTransformHelper, instead of using the global
metrics instance. Otherwise, the metricsPrefix defined in the parsoid
settings is ignored.
Change-Id: Ie85f2306e8b0f123b9fdd737faffdd85117015b1
If a render ID is given via the use-cache parameter, but the key is not
found in the parsoid stash, look at the most recent known rendering of
the revision, and use it if it matches the render ID.
This patch moves the responsibility for looking up RevisionRecords and
PageRecords into ParsoidOutputAccess. This way, callers only need to
have a PageIdentity, and optionally a revision ID.
Bug: T318395
Change-Id: I1aa5b0fd9fb1acaa2544d5a58125fa3810a0eb39
Parsoid needs the original rendering in order to apply
selective serialization (selser). The page/{title}/html endpoint
can stash the rendering, and now the transform endpoint can make use
of the stashed rendering.
Bug: T310464
Change-Id: Ia58043ed3aa1eb12731d82aa87606c82ec63f663
The HtmlInputTransformHelper is intended to provide code sharing
between VisualEditor's DirectParsoidClient and the ParsoidHandler
base class used by TransformHandler.
Bug: T310376
Change-Id: I9c15f075cfc5f198e290758fc23d25990b47a185
Parsoid supports other source formats besides wikitext.
This patch improves support for non-wikitext content by removing
assumptions about the source type.
Change-Id: I5480ff200a93026cea7f1542e12834b06ac6f730
This also moves the creation of PageConfig from HTMLTransformFactory
into HTMLTransform, to ensure all relevant info, particularly the
page language, is known.
Change-Id: Id354862d6497816e0c007b9cb3b0d183c9d4b719
This patch creates a patch to cover the aforementioned code path for
when it works correctly and when it throws.
Change-Id: I4b3d9f280a0977d3811afb768824da302673e659
By splitting the setOriginalData methods into several setters, we remove
any knowledge about the structure of the request body from HTMLTransform.
It also allows us to be specific about which data to operate on.
This also removes the concept of page bundles from the public interface
of HTMLTransform. PageBundle objects are used only internally.
Change-Id: If97a74ce251f281b7d980928a01b764d6ec0d0a4
This patch moves remaining transformation logic to a renamed (from
HTMLTransformInput -> HTMLTransform) class. Also, the HTMLTransform
class is moved to the correct directory, hence namespace (including
tests).
Some data files have been copied over to it's own sub-directory in
the correct place since HTMLTransformTest needs it. ParsoidHandler
class is fine where it is because its operation is what happens in
the REST land.
NOTE: The 2 remaining methods moved into HTMLTransform are the last
ones we intended to move into this class to make the refactoring of
html2wt() method complete in this context.
Change-Id: I8929931e1b0acf247abe9d826eef57f3e0d4e132
When refactoring, we were assuming that if the input is a page bundle,
it will always contain original HTML. That is not the case. We need to
make use of the supplied data-parsoid array even if there is no original
HTML given.
Change-Id: Ida8cbfcaac059af8902db9560a3ad6884e8b1790
This is a regression test for an issue that broke parsoid's roundtrip
tests. The issue itself was already fixed on the master branch.
Change-Id: Ia7edb26a149231fbfc43f4edb8185304341b19ca
This moved the logic for applying page bundle data into the getters that
return DOM elements. It also makes the application of version
downgrades implicit.
NOTE: This patch changes the expected value of one of the phpunit tests
to a version that has no closing </div> tags. This appears to be the
original and expected behavior, per the corresponding test in the
parsoid extension's Parsoid.js test suite.
Change-Id: If2d7b06d8ba92fb63e6955ec7587ed4aea557251
* Several unit tests had lost information about the input format
being a pagebundle format.
* Fix the input format for a number of tests and a test expectation
in one case.
I removed a number of spurious comments which were misinterpreting
why the pagebundle format was being used. If the input payload is
in pagebundle format, the specified format should match that.
In integration tests, the API endpoint implicitly provides this
format information. But, in unit tests, this should be done explicitly
which would also be the case for internal service object callers.
Change-Id: If3d3c0ad2ee1fe19f4c1590caa43bea4340cc08c
Moving lazy initialization logic into HTMLTransformInput to clarify flow
of information.
DEPLOY: Mention that we've fixed $htmlSize and it will affect metrics
graphs when this roles out.
Change-Id: If09afc4b933ec99561ac9e4d53383bd42a856eaa
We were only testing the offset type mismatch case with a non-standard
offset type in the 'original' data structure. This cause us to miss a
bug. This adds a test that has a non-standard offset type coming from
the request attributes.
Change-Id: Icac28b3f8c85bc594e1cae42a4ee407f0d54a19b
