Commit graph

18 commits

Author SHA1 Message Date
daniel
1271faa381 Move access to the page bundle into ParsoidOutputAccess
This isolates ParsoidHTMLHelper from the internal of
ParsoidOutputAccess. The corresponding test cases were changed to use a
mock ParsoidOutputAccess, and to not test the behavior of
ParsoidOutputAccess.

Bug: T301371
Change-Id: Id693fae2264f15e5d35f28acc5adc4239b2ae24f
2022-06-28 11:49:36 +02:00
Derick Alangi
1854fb02d9 Storage: Warm parsoid parser cache with parsoid outputs
This patch introduces a ParsoidOutputAccess service for
getting parsoid outputs and warms the cache with pregenerated
outputs.

It also introduces a config variable in ParsoidCacheConfig that
is turned off by default for controlling the cache warming.

Bug: T301371
Change-Id: I6152c42ea765d94093d8d62598b1b4278314adec
2022-06-28 09:05:41 +00:00
Derick Alangi
270699ec34 Configure caching parsoid output per wiki based on threshold
Cache the parsoid outputs only if a certain time is exceeded on
parse and consider the parse operation within this time limit as
not expensive per that wiki and not cache the parsoid output at all.

Bug: T308588
Change-Id: I7793b77feab13400ccd04343e7878ad701f5e6a7
2022-06-16 11:42:06 +01:00
daniel
6955380fbe Add rate limiting to ParsoidHTMLHelper
Bug: T267991
Change-Id: I52a83e7d3bdb0bcde59160e2d193f06908fda3d4
2022-06-15 13:40:56 +02:00
Subramanya Sastry
d3c3abcd22 Unbreak ParsoidHTMLHelperTest: Fix call to ParsoidHTMLHelper constructor
Change-Id: I17af4240ac81c6234c213c7a92465354a9942201
2022-06-02 17:26:24 -05:00
Derick Alangi
d62f97d5e0 Rest: Return different eTags for different output modes
This patch enables the response from PageHTMLHandler and
RevisionHTMLHandler to have different eTags for different
output modes and varying flavors.

Before, the only difference we got was when the stashing
option is set or not, but we need more flavors.

Bug: T308744
Change-Id: I2e9679e46a31955a2106a52af4eb612b32799c8c
2022-05-25 11:15:47 +00:00
Derick Alangi
13f6ec9e1b Rest: Migrate parsoid stashing logic from RESTbase
Add stash option to /page/html & /revision/html endpoints.
When this option is set, the PageBundle returned by Parsoid is
stashed and an etag is returned that can later be used to
make use of the stashed PageBundle.

The stash is for now backed by the BagOStuff returned by
ObjectCache::getLocalClusterInstance().

This patch adds additional data to the ParserOutput stored in ParserCache.
Old entries lacking that data will be ignored.

Bug: T267990
Co-Authored-by: Nikki <nnikkhoui@wikimedia.org>
Change-Id: Id35f1423a69e3ff63e4f9883b3f7e3f9521d81d5
2022-05-23 17:28:29 +01:00
Derick Alangi
1618bbd671 Add data-parsoid data to ParserOutput for caching
NOTE: This changes the HTML returned by the endpoint!
It will now include the id="mwXYZ" attributes needed to
later map to data-parsoid entries.

Bug: T268205
Change-Id: I0a29434b996cc289eb67083e62bd6f1ad750cb4d
2022-05-16 15:06:15 +00:00
Subramanya Sastry
8aae1a0b21 Fix error message in test assertion
Change-Id: Id7604d9af623ffdb77b16960e8b9ca893d368963
2022-03-09 16:29:25 -06:00
Reedy
6e29611642 Remove or replace usages of "sanity"
Still some more to go...

Bug: T254646
Change-Id: Ia117f01e443c35b4765f3275cab4f2707e1be96f
2021-11-21 16:42:31 +00:00
Petr Pchelko
4ca16e8d08 Eliminate use of Title object in REST infrastructure
Change-Id: I585f0f23cac5f6dc2a4879f69f7b83828fda3dd3
2021-05-05 18:54:58 -07:00
Petr Pchelko
f642215aed Convert ParserCache to PageRecord
ParserOptions not updated cause they depend on Title::getLanguage
implementation.

Tests converted to not require a DB anymore. Can't be proper unit
tests yet due to globals in ParserOptions and fake time hacks,
but exec time does go down from 70 seconds to 9 seconds.

Page content model is still emitted in the metrics since
it was considered useful. Should be removed when we get
something like a page type concept.

Change-Id: Ib16fd0b5b87ffc3cb4d21f4aa43d1203cb7206d2
2021-04-02 21:14:54 -06:00
Thiemo Kreuz
2f66b3754f tests: Remove @param docs from test code that just repeat the signature
These are not only 100% identical to the actual code, but also:
* It's error-prone. Some are already wrong.
* These test…() functions are not meant to be called from
  anywhere. What is the target audience for this documentation?
* There is a @dataProvider. What such @param tags actually do is
  document the provider, but in an odd place. Just looking at
  the provider should give the same information.
* The MediaWiki CodeSniffer allows to skip @param when there is
  a @dataProvider, for the reasone listed.

Change-Id: I0f6f42f9a15776df944a0da48a50f9d5a2fb6349
2021-01-21 03:41:23 +00:00
daniel
637f630fe9 Implement caching for old revision HTML endpoint
Bug: T269663
Change-Id: I2d17ec37d25f3a6e1c4836c05576bf0fabb7d429
2020-12-15 23:40:08 +01:00
Petr Pchelko
1162411d7f Make /page/{title}/html emit etags in RESTBase format
RESTBase used to emit ETag in the `"<rev_id>/<render_id>" format.
For the benefit of the clients, preserve the formar.

Render ID is a UUIDv1 uniquely identifying the ParserOutput.
In future it would be used as a stashing key for stash deduplication.
At this time I decided to just attach the render ID as extension data
to our fake ParserOutput. Once we integrate Parsoid more into core,
we will likely move it into a ParserOutput property, or even
replace CacheTime::mCacheTime with a UUIDv1, but it's too early for that.

Bug: T268234
Change-Id: Ie604e9c98021d59eb1a17ca65f227e8f234a45be
2020-12-09 16:36:07 -06:00
Daniel Kinzler
3bc61324b9 Re-Apply "Extract helper classes from PageHTMLHandler"
This reverts commit d51a697e13.

Reason for revert: Let's try this again...

Change-Id: Ie0218adff95576c972ff4c1d51cadd02f41eba3e
2020-12-07 16:59:29 +00:00
Subramanya Sastry
d51a697e13 Revert "Extract helper classes from PageHTMLHandler"
This reverts commit b98f7a6fc1.

Reason for revert: Breaks Parsoid CI but doesn't seem to run on core patches?

Change-Id: I1eaf1495dce6f6ba78093aacb9475a023a2aabfa
2020-12-02 23:32:27 +00:00
daniel
b98f7a6fc1 Extract helper classes from PageHTMLHandler
This extracts two helper classes from PageHTMLHandler:
* PageContentHelper for accessing page content. This replaces the
  LatestRevisionContentHandler mase class.
* ParsoidHtmlHelper for generating HTML from wikitext using parsoid.

The idea is to decouple the functionality from the REST handlers, so we
can easily mix and match functionality to create a handler for the
new per-revision HTML endpoint.

Bug: T267981
Bug: T267982
Change-Id: I3226833d12e51c959712d642b0195de1fe1ef979
2020-12-02 18:08:12 +00:00