Commit graph

178 commits

Author SHA1 Message Date
Derick Alangi
1afd52e3e4 REST: Move Helper classes to their own namespace
Mixing Handlers with Helpers doesn't look nice for consistency
reasons. Helpers should be in their own place (grouped) in the
Handlers directory as they're really "helpers for the handlers".

Change-Id: Ieeb7a0a706a4cb38778f312bfbfe781a1f366d14
2023-01-16 21:16:09 +01:00
Derick Alangi
d51522dfd3 ParsoidOutputAccess: Mark dummy parser output as non-cacheable
Bug: T311728
Change-Id: I55f153d21f91ef93fe5c788fd054fa481fc2ab10
2023-01-10 12:43:30 +01:00
jenkins-bot
c9fea125c7 Merge "PageContentHelper: Remove back-compat code for VisualEditor" 2023-01-10 11:30:50 +00:00
Derick Alangi
cdd49f7536 ParsoidOutputAccess: Completely handle unsupported content models
Pages with content models not supported by Parsoid before already
had dummy parser output written to PC but we still got an internal
server error because the output didn't have a render key.

This patch fixes the issue and when we try to render page with
unsupported content models, we get the dummy parser outputs.

Bug: T311728
Change-Id: I49ebbfc0475fb296f2a906ce7dce237641fb375b
2023-01-10 08:38:51 +00:00
Derick Alangi
467fbd3f6c PageContentHelper: Remove back-compat code for VisualEditor
Was introduced in I10af85b2da96568cfffd03867d1cb299645fb371 for
backward compatibility with VisualEditor extension.

Change-Id: I180c5d95d1056cd2b5d2126b0622c5316dbf9c7b
2023-01-10 08:38:06 +00:00
DannyS712
1f8e5bcefe DummyServicesTrait: add content handler factory
Simplify mocking common cases of the content handler factory
only support specific content models.

Change-Id: Ib7f4099a2b1655b1bfb4d90f03105c27d17042ed
2023-01-02 01:09:42 +00:00
jenkins-bot
4fde3b96de Merge "ParsoidOutputAccess should support all models that serialize to wikitext." 2022-12-16 19:36:18 +00:00
daniel
e1c3af9177 ParsoidOutputAccess should support all models that serialize to wikitext.
The motivation is to restore parsoid support for the content models
defined in the Proofread extension.

Bug: T246403
Change-Id: I33d269e42fede28139f7c923504326a77d11ee13
2022-12-16 12:20:10 +01:00
Derick Alangi
ce8e5f1549 Introduce HtmlMessageOutputHelper for system messages
This introduces an interface HtmlOutputHelper that is implemented
by both HtmlMessageOutputHelper or HtmlOutputRendererHelper based
on the page we're dealing with.

Bug: T323558
Change-Id: I1fb8dcc5cc05ce3f32f3c1862b88045f1c8e612b
2022-12-16 11:49:56 +01:00
jenkins-bot
630925a09d Merge "HtmlOutputRendererHelper: test that parser cache is used." 2022-12-15 18:22:11 +00:00
daniel
91c1380d1e HtmlOutputRendererHelper: test that parser cache is used.
This protected against the performance regression that was addressed in
abec307409.

Change-Id: Ie5f7c2c713a10eec19bc19e0ba709f36599fb291
2022-12-15 17:56:14 +00:00
jenkins-bot
4b2c11a85a Merge "Parsoid: return 404 when page is deleted." 2022-12-15 12:09:47 +00:00
jenkins-bot
dba0bf3877 Merge "add redirects to page/history and link endpoints" 2022-12-15 11:37:43 +00:00
daniel
6decc0ba09 Parsoid: return 404 when page is deleted.
If HtmlOutputRendererHelper can't produce output because the page
doesn't exist, don't crash. Return a 404 instead.

Bug: T325205
Change-Id: I6a713025e2eb447ed61de8949c820bd9cda7594e
2022-12-15 12:13:57 +01:00
Derick Alangi
fe091a7cad Parsoid: Default parsoid version to "0.0.0" for unsupported models
When parsoid was dealing with content for content models that it does
not support, the corresponding pagebundle used `null` as the version
which will break when converting the it to JSON.

This patch fixes this by just using a version which is not specified
like 0.0.0

A better fix would be for page bundle to not explode when the version is
set to null. The other real fix is for Parsoid to not ask for rendering of
pages that are not wikitext.

Bug: T325137
Change-Id: Iff1ce432d1b2d30f3f74c53a0602c11034db5874
2022-12-14 13:07:18 +01:00
daniel
4f22f967c5 Parsoid: implicitly enable linting in API endpoints
Logging linter data should be enabled automatically by
HtmlOutputRendererHelper.

This change enables linting data for requests coming in via the
v1/page/{title}/html endpoint.

Change-Id: Idafd29784ec712547e36fea88a8c159784b97f2b
2022-12-13 13:35:06 +01:00
daniel
5559d2d471 Parsoid: Enable lint data and parser cache together
The previous attempt at restoring call to the ParserLogLinterData hook
had the undesirable effect of bypassing parser cache. This change
optionally enables the call to the lind data hook without disabling
parser cache.

This patch us working under the assumption that we only need to log lint
data for canonical parses.

Follow-Up to I1f69498ef75

Change-ID: I39ab54ca6e9f9a6a58b59cdd6feea07638fc908f
2022-12-12 22:57:07 +01:00
jenkins-bot
374d578707 Merge "ParsoidOutputAccess: generate dummy output for unsupported models." 2022-12-12 20:13:31 +00:00
jenkins-bot
947c3ddba0 Merge "ParsoidOutputAccess: only cache output for wikitext" 2022-12-12 20:09:30 +00:00
daniel
75d6892134 ParsoidOutputAccess: generate dummy output for unsupported models.
While RESTbase insists on getting Parsoid renderings for any content
model, don't waste CPU cycles trying to render garbage. Just output
dummy content. Nobody should ever see it.

Bug: T324711
Change-Id: I407171a5f515b594603b287a7a9e49f54e837161
2022-12-12 19:57:31 +00:00
jenkins-bot
b340554a4d Merge "ParsoidHandler: test wt2html with old revision" 2022-12-12 13:22:36 +00:00
daniel
5f2026c31c ParsoidHandler: test wt2html with old revision
Update the test for wt2html to assert that it works properly with an old
revision.

Bug: T324801
Change-Id: Ia2a7e28cd999712b1bd890eed48d0a5de931700f
2022-12-09 19:33:18 +01:00
Amir Sarabadani
a1b4699fea Reorg: Move MagicWord related files to under parser/
This is approved as part of T166010 RFC.

Bug: T321882
Change-Id: Ia4498c0a20e38a6a288dc14065ea8242c84fbc49
2022-12-09 13:48:35 +01:00
msantos
63deedc94c add redirects to page/history and link endpoints
* Add normalization redirects for page/history, page/history/counts
and links/language endpoints

Bug: T301372
Change-Id: I731fbfdcfb538c3e8a5dd32349ebaa0e642bf338
2022-12-09 12:17:38 +00:00
daniel
9842811e75 ParsoidOutputAccess: only cache output for wikitext
ChangeProp is currently requesting a parsoid parse for all page updates,
regardless of content model. Parsoid renderings of non-wikitext content are
unusable, so we shouldn't bother the parser cache with them. This is
especially true for wikidata items.

Bug: T324711
Change-Id: I6f6325f2b8581dfcc9a8bcd97281ccf4caa7e8f1
2022-12-08 18:29:24 +00:00
Abijeet
803092d4af tests: Remove unnecessary override to use pig-latin
Pig latin is enabled by default since
Ia80ad33cbf5e311fa8b84bd765a8df8d156f4c38

Change-Id: I0cd922bb0ee1fd7bce2ced2eacbdb6ed25ada7d8
2022-12-08 17:52:00 +05:30
jenkins-bot
1b3d4d17d4 Merge "Throw a 400 when asking parsoid to render an unknown content model." 2022-12-07 22:44:45 +00:00
daniel
a79b73e722 Make parsoid accept all content models.
This allows parsoid render anything, even if the output is garbage.
This is a quick fix pending the real solution (T311728).

Bug: T324711
Change-Id: If4e4eb8582ab8a62f592394820b30c1b28fb1216
2022-12-07 22:23:54 +00:00
daniel
d34e73b0c1 Throw a 400 when asking parsoid to render an unknown content model.
Bug: T324711
Change-Id: I0a78e5c57e2b8449b393bccc86148aee4ad87bc8
2022-12-07 23:09:58 +01:00
Derick Alangi
e1344b76cd Fix typo (hmtl) to html where necessary
Change-Id: If837968f402c71813121754b8fbdb5519c3dae34
2022-12-07 17:11:48 +01:00
jenkins-bot
bcb6cc5bbb Merge "HtmlOutputRendererHelper: fix semantics of getRevisionId" 2022-12-07 12:33:21 +00:00
jenkins-bot
ca84394a1d Merge "Use services in WikitextContentHandler" 2022-12-07 11:43:18 +00:00
daniel
9ff8edfa1e HtmlOutputRendererHelper: fix semantics of getRevisionId
getRevisionId is documented to return 0 for fake revisions, but it was
returning 0 for the current revision as well. This patch makes a clear
distinction, with 0 meaning current (like elsewhere in the code), and
null meaning a fake revision.

This patch includes a fix for redirect handling in ParsoidHandler::wt2html.
This fix is needed here because it previously relied on getRevisionId()
to return the actual revision ID; this would fail, since getRevisionId()
will return 0 when the current revision of a page is requested.

Change-Id: I33d1ab54023c6ac96c6bb5e4750b980e381cb464
2022-12-06 23:06:25 +01:00
daniel
090ec5777d Use services in WikitextContentHandler
Change-Id: I626b5ee9a070ad3a97ab9ac9f44cb7003d68bf13
2022-12-06 15:44:40 -05:00
jenkins-bot
f253617b73 Merge "HtmlOutputRendererHelper: fix etag tests" 2022-12-06 14:07:45 +00:00
daniel
42e55bcd6f HtmlOutputRendererHelper: fix etag tests
The test cases getETag and getLastModified where checking the method's
return values against themselves.

Change-Id: I13c5abfdd07373795ca9b661d812ec0eed11f766
2022-12-06 13:23:44 +00:00
Arlo Breault
dc02ef2500 Permit Parsoid minor version bumps
Loosening the version checks in the integration tests is akin to what
was done for api-testing in I6d7db6a05c48de8a57f83e4c8af38ab50271297a
and I317ce587e62f9e94bbafbdabac64156237c4f1e3.

The tightly coupled tests were added in Ieb4b41375d521893f95e2fcb5f4984e7b5a2364c

The change to the condition when original html needs downgrading was
exposed by the test added in Ic3cc3a598f32cad6122964cb8a7376a56be9129f
though not exercised because edited and re-parsed html had the same
versions.  Without it, in-flight edits on wikis that might request a
re-parse (think private wikis) when a new semantically equivalent
version is deployed could fails due to trying to find a downgrade path
and none being there.

Needed-By: Iabab0c093dcb21e28c643be6e85cf1a7b54cd999
Change-Id: I33e70df750c6d4b082281fdc8bacdea72662832a
2022-12-05 19:51:38 -05:00
jenkins-bot
5a64f38dfe Merge "[Fix] ParsoidHandler: use HtmlOutputRendererHelper in wt2html" 2022-12-05 19:21:31 +00:00
jenkins-bot
cca97bcad7 Merge "[Re-apply] ParsoidHandler: use HtmlOutputRendererHelper in wt2html" 2022-12-05 19:21:23 +00:00
Daniel Kinzler
f36a28ff21 [Fix] ParsoidHandler: use HtmlOutputRendererHelper in wt2html
Fixes the reason for reverting Ie430acd0753880d88370bb9f22bb40a0f9ded917:

The issue was that with my patch, the transform/wikitext/to/html started
ignoring the offsetType field in the body. So the offsetType used in the
response (or stashed data) would always be 'byte'.
But the roundtrip-test.js scripts requests 'ucs2'.

This causes an error when sending the HTML and data-parsoid back to
transform/html/to/wikitext, again with offsetType:'ucs2': the offsetType
embededed in data-parsoid will be byte, and the mismatch causes a 400
to be returned. This broke the roundtrip-test.js script.

The fix is to no ignore the offsetType specified in the request body.

Change-Id: Ief721c23ed9a57d781cfdac625a62113f22f87a5
2022-12-05 18:49:30 +00:00
Daniel Kinzler
5cb388455b [Re-apply] ParsoidHandler: use HtmlOutputRendererHelper in wt2html
This restores change Ie430acd0753880d88370bb9f22bb40a0f9ded917.
This reverts commit ab6baad1a5.

NOTE: Also needs the patch the fixes the original reason for the
revert: Ief721c23ed9a57d781cfdac625a62113f22f87a5

Change-Id: Ic48db1b5fdff1dfd4f2d2643d64252e5fc721e79
2022-12-05 18:43:51 +00:00
msantos
fb5c29e2c0 Follow redirects for page/{title} formats source/bare
* Share logic previously implemented for html/with formats through
a trait class

* source/bare formats doesn't execute a temporary redirect. the
JSON body will contain a key "redirect_target" instead if a wiki
redirect is found

* Introduce PageRedirectHandlerTest to test redirect logic shared
between multiple handlers

* Move Handler instatiation to HandlerTestTrait

* Update api-testing tests in Update.js

Change-Id: Id66e33e19adabdb3c9621eaea4a5d441f23edafd
2022-12-02 13:22:14 -03:00
Daniel Kinzler
ab6baad1a5 Revert "ParsoidHandler: use HtmlOutputRendererHelper in wt2html"
This reverts commit e82f11c246.

Reason for revert: Breaks parsoid CI

1) Parsoid round-trip e2e testing with MW REST endpoints
     rt-testing e2e:
     AssertionError: expected 1 to equal 0
     + expected - actual
     -1
     +0

     at Context.<anonymous> (tests/api-testing/RoundTrip.js:59:10)
     at processTicksAndRejections (internal/process/task_queues.js:95:5)

Change-Id: Ib94f964c2717885f777c1fe0c9c443cd6a5ed3ae
2022-12-01 21:17:34 +00:00
daniel
e82f11c246 ParsoidHandler: use HtmlOutputRendererHelper in wt2html
NOTE: This causes Parsoid output to be written to the parser cache.
This should be unconditional in the future, but for now it is
controled by wgTemporaryParsoidHandlerParserCacheWriteRatio.

This change affects the following endpoints that use the wt2html method:
* /coredev/v0/transform/wikitext/to/html in core
* /{domain}/v3/transform/wikitext/to/html from parsoid
* /{domain}/v3/page/html/{title} from parsoid

The /v1/page/{title}/html endpoint is not affected, since it
doesn't use wt2html, but has always been using HtmlOutputRendererHelper
directly.

Bug: T322672
Depends-On: Ic37f606bb51504c8164d005af55ca9a65f595041
Change-Id: Ie430acd0753880d88370bb9f22bb40a0f9ded917
2022-12-01 10:14:49 +00:00
daniel
c6b4a18573 HtmlOutputRendererHelper: Add edit flavor
This adds support for an "edit" flavor to HtmlOutputRendererhelper,
which causes data-parsoid attributes to be present in the HTML output.

In addition, this adds the ability to request HTML output using a
specific profile versionThis adds support for an "edit" flavor to
HtmlOutputRendererhelper, which causes data-parsoid attributes to be
present in the HTML output.

In addition, this adds the ability to request HTML output using a
specific profile version..

Change-Id: I5c21e0358da6a8915ddfe574fb2d13e469f625c5
2022-11-28 12:12:15 +01:00
daniel
93015579a1 HtmlOutputRendererHelper: add more convenient setters
REST helper objects should be geared towards accepting input directly
from an HTTP request. As such, they should offer setters that take
string values. And native representation of things like page titles,
languages, or content objects should be done implicitly by the helper.

Change-Id: I9b81cad4d5cc575e7c5283035e385ac0457e8059
2022-11-28 12:07:17 +01:00
daniel
c6a0d433ec HtmlOutputRendererHelper: allow parser cache to be disabled.
This is needed so we can ramp up parser cache writes in a controlled
manner.

Bug: T322672
Change-Id: I7d97c9e2d4009029dc64f9c0a369f68098185520
2022-11-28 09:43:12 +00:00
Derick Alangi
506ca39387 Rest: Allow fetching content of pages with system messages
NOTE: some phan suppressions removed, no longer needed now.

Bug: T317017
Change-Id: Iccbffb502eeace841f86ddb1987e77be50003973
2022-11-23 17:00:00 +01:00
thiemowmde
0b80e9ebcc Fix incomplete ITextFormatter mocks
Otherwise the mocked getLangCode() method returns null, which is not
allowed any more in PHP 8.1.

Bug: T289926
Required-For: I7e026cca216aba24ee5d5662b6fca322b3cec9ae
Change-Id: I178def7f03a44f6b49cdb461d9ab340e1c89517f
2022-11-21 10:00:57 +01:00
daniel
2ec1791d40 Introduce PageRestHelperFactory
This allows extensions like VisualEditor to safely instantiate REST
helper objects. It also reduces the number of services that need to be
injected into REST handlers from route definitions.

Change-Id: I10af85b2da96568cfffd03867d1cb299645fb371
2022-11-21 07:23:26 +00:00