This touches various production classes and maintenance scripts.
The code should do the exact same as before. The main benefit is that
the syntax avoids any repetition.
Change-Id: I5c552125469f4d7fb5b0fe494d198951b05eb35f
Since I72c5e6f86b7f081ab5ce7a56f5365d2f75067a78 it is part of the
contract of ContentRenderer::getParserOutput() that the render ID (and
other cache parameters) will be set when it returns.
(ContentHandler::getParserOutput() can set them even earlier if it has
custom content-based overrides.) We had a lot of temporary
backward-compatibility code "later" in the parse process to try to close
the barn door if some code path "forgot" to set them, but these are
unnecessary now.
This patch removes that backward-compatibility code in ParsoidParser;
there is similar remaining code in ParserCache etc. which can be
addressed in follow ups.
(For compatibility we do have to temporarily copy the render ID code
inside ParsoidOutputAccess::parseUncachable, but that class is
deprecated and will be removed.)
The HtmlOutputRendererHelper path which used to call
ParsoidParser::parseFakeRevision() is now replaced with a codepath that
goes through RevisionRenderer. In order to maintain the same behavior
of the ParsoidHandler, we have also added 'useParsoid' handling to the
JsonContentHandler. This support can perhaps be deprecated eventually.
Bug: T350538
Change-Id: I0853624cf785f72fd956c6c2336f979f4402a68f
Instead of creating a half-initialized helper and later calling ::init,
provide all the information necessary for the helper in the constructor.
This is facilitated by the fact that there already exists a factory
class, PageRestHelperFactory, which holds all the services required.
This affects:
* HtmlOutputRendererHelper::init()
* HtmlMessageOutputHelper::init()
* HtmlInputTransformHelper::init()
Change-Id: I1e1213597c6be012f2bc024c2b370c968ff3b472
This removes the last use of ParsoidOutputAccess in core, allowing it
to be deprecated and eventually removed.
Bug: T367074
Bug: T317018
Change-Id: Ica2c880e2e7c2b126aaea66a3e4be460b3f2234f
Changes to the use statements done automatically via script
Addition of missing use statement done manually
Change-Id: Id9f3e775e143d1a17b6b96812a8230cfba14d9d3
Mostly used find-and-replace:
Find:
/\*[\*\s]+@var (I?[A-Z](\w+)(?:Interface)?)[\s\*]+/\s*(private|protected|public) (\$[a-z]\w+;\n)((?=\s*/\*[\*\s]+@var (I?[A-Z](\w+)(?:Interface)?))\n|)
Replace with:
\3 \1 \4
More could be done, but to keep this patch reasonably sized, I only
changed the most obvious and unambiguously correct cases.
In some cases, I also removed redundant doc comments on the
constructor, and re-ordered the properties to match the constructor.
Change-Id: Ifa710fdf4d8d44a2d7244798b787a1b2a58c35a7
One more step in gradually replacing uses of ParsoidOutputAccess with
ParserOutputAccess. This is mostly just inlining the code from
ParsoidOutputAccess (which already uses ParserOutputAccess internally)
and then removing dead code.
Change-Id: I87d6568d9dc71bc11f406ea3c31ce86e01667e05
HtmlOutputRendererHelper was logging an error when the user doesn't have
persmission to access a revision. It should just send a 403 (or 404,
depending on the case).
BUG: T356157
Change-Id: I5ed2f85edbd1350ef5872238be4cecccdec9bd4b
The new trait provides conveniance methods for logging and throwing
errors based on StatusValue instances.
Change-Id: Id7cbacb744bee79cd8a6f61291a26b36e6243053
HtmlOutputRendererHelper should not crash hard if the ParserOutput has
no language set. ParserOutput may come from a variety of places, we
should be lenient about it not having a language.
However, we should try harder to actually set a language on ParserOutput
if we have one available. So this also updates
PageBundleParserOutputConverter to keep the ParserOutput's language in
sync wit the language header in the PageBundle.
Bug: T349868
Bug: T353689
Bug: T359426
Change-Id: I2edf20dc3b199e22cda2f32bc858c21ca7d8f4bd
We can go from a Status to a MessageValue like this.
There could be a better way of doing this, but it's still better
than rendering the message while we don't know the desired language.
Change-Id: Ia1e34fa41a3013cc0dcad65f1b5cdad1d5fbb4c6
When HtmlOutputRendererHelper gets an error from Parsoid, we return a
status 500. Before we do so, we should generate a log entry, so we can
be aware of these errors occurring.
Bug: T356157
Change-Id: I632272e71d24aa1a4ad27b2dcece81f8b10a7e49
PoolCounterWorkArticleView was not designed for use by all callers of
getParserOutput. It provides stampede protection but does not generally
prevent duplicate concurrent parsing, and it may result in stale cache
entries being returned to the caller. This is acceptable for page views,
but not other use cases like editing or updating secondary derived data.
Bug: T352837
Change-Id: Ie532c17e5b86e8e1adbb57ecd5c5c6405b83bf8f
When using the accept-language header to determine the desired target
variant, use only a single language code from the header, instead of the
entire string. Using the entire string was causing failures in ETag
processing, since the accept-language header may contain spaces, while
the ETag may not.
Note that for now, we just use the first language code. In the future,
it would be nice if we could find the supported target variant that is
most preferred in the header.
Bug: T350852
Change-Id: I32af29627f68eafff6f097e2fc17723d2ebb39fa
ParserOutput::getText() is not a simple getter, but does
transformations on the "text" of the ParserOutput; the simple getter
is named ::getRawText().
To maintain consistency, rename ParserOutput::setText() to
::setRawText() and the property name ParserOutput::$mText to
::$mRawText so future readers are not confused.
The JSON property name as it appears in the serialized ParserCache
is left as 'Text' so that we don't have any forward- or backward-
rollback issues.
Change-Id: I3ef34814ab9473cc70d0a6806e8c5a4a02b73491
Set the render ID for each parse stored into cache so that we are able
to identify a specific parse when there are dependencies (for example
in an edit based on that parse). This is recorded as a property added
to the ParserOutput, not the parent CacheTime interface. Even though
the render ID is /related/ to the CacheTime interface, CacheTime is
also used directly as a parser cache key, and the UUID should not be
part of the lookup key.
In general we are trying to move the location where these cache
properties are set as early as possible, so we check at each location
to ensure we don't overwrite a previously-set value. Eventually we
can convert most of these checks into assertions that the cache
properties have already been set (T350538). The primary location for
setting cache properties is the ContentRenderer.
Moved setting the revision timestamp into ContentRenderer as well, as
it was set along the same code paths. An extra parameter was added to
ContentRenderer::getParserOutput() to support this.
Added merge code to ParserOutput::mergeInternalMetaDataFrom() which
should ensure that cache time, revision, timestamp, and render id are
all set properly when multiple slots are combined together in MCR.
In order to ensure the render ID is set on all codepaths we needed to
plumb the GlobalIdGenerator service into ContentRenderer, ParserCache,
ParserCacheFactory, and RevisionOutputCache. Eventually (T350538) it
should only be necessary in the ContentRenderer.
Bug: T350538
Bug: T349868
Followup-To: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
Change-Id: I72c5e6f86b7f081ab5ce7a56f5365d2f75067a78
A LogicException is very generic and doesn't mean much. An
InvalidArgumentException is also a LogicException, but more
specific: A method was called (for whatever reason – bad code,
bad user input – we don't know) with an invalid, unsupported
argument. This is exactly what's going on in these cases.
BadMethodCallException does have a confusing name and is often
misused because of this. It's documented as "thrown if a callback
refers to an undefined method or if some arguments are missing".
That's something else and not what's going on in these cases.
Change-Id: Id446227f578ba701e22acd5e530ffb795e76c147
This introduces RestAuthorizeTrait to ensure proper error reporting
after calls to Authorizer methods to avoid misleading error reports,
see T350117 and T350202.
This reverts commit e047668d9f.
This restores change 701ff30193.
Change-Id: I617cb7ba24a1614c39e2b1072888f0ee7b3127e3
Per T310476, it looks like the Authority interface now supports
rate limiting, so we can just use that instead of a heavy full user
object.
Possible followups
==================
More can be done to make all consumers of the HTML helper's `init()`
method to inject Authority instead.
This patch is needed for the work happening in I08ebea5e8a601f161f.
NOTE: This is technically not a breaking change as the Authority
interface is implemented by both UserAuthority and User classes,
so passing either is fine so consumers passing a full user object
should still work even though we changed the signature of a public
method in HtmlOutputRendererHelper.
Change-Id: I025cd83cc81f73ded861fcab943ba3b942d7c390
* Parsoid REST API which considers both title and revid will soon
be made internal. The core REST API only has endpoints where either
the title OR the revid is provided and is not subject to this issue.
* This patch ignores page id mismatches and simply uses the revision
page id where the mismatch is detected. This is only supported for
ParsoidHandler which sets the lenient revision handling while fetching
the HtmlOutputRendererHelper.
For these API requests, the output is not cached.
* Local testing shows that this fixes the issue.
* Added new phpunit tests to ParsoidHandlerTest to verify expectations.
Also verified that disabling this fix fails that test.
Bug: T349235
Change-Id: I2f4a4a644710ee1e3894e6dc6a066eb37846bdfd
* Updated ParserOutput to set Parsoid render ids that REST API
functionality expects in ParserOutput objects.
* CacheThresholdTime functionality no longer exists since it was
implemented in ParsoidOutputAccess and ParserOutputAccess doesn't
support it. This is tracked in T346765.
* Enforce the constraint that uncacheable parses are only for fake or
mutable revisions. Updated tests that violated this constraint to
use 'getParseOutput' instead of calling the parse method directly.
* Had to make some changes in ParsoidParser around use of preferredVariant
passed to Parsoid. I also left some TODO comments for future fixes.
T267067 is also relevant here.
PARSOID-SPECIFIC OPTIONS:
* logLinterData: linter data is always logged by default -- removed
support to disable it. Linter extension handles stale lints properly
and it is better to let it handle it rather than add special cases
to the API.
* offsetType: Moved this support to ParsoidHandler as a post-processing
of byte-offset output. This eliminates the need to support this
Parsoid-specific options in the ContentHandler hierarchies.
* body_only / wrapSections: Handled this in HtmlOutputRendererHelper
as a post-processing of regular output by removing sections and
returning the body content only. This does result in some useless
section-wrapping work with Parsoid, but the simplification is probably
worth it. If in the future, we support Parsoid-specific options in
the ContentHandler hierarchy, we could re-introduce this. But, in any
case, this "fragment" flavor options is likely to get moved out of
core into the VisualEditor extension code.
DEPLOYMENT:
* This patch changes the cache key by setting the useParsoid option
in ParserOptions. The parent patch handles this to ensure we don't
encounter a cold cache on deploy.
TESTS:
* Updated tests and mocks to reflect new reality.
* Do we need any new tests?
Bug: T332931
Change-Id: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
This reverts commit cbde6b69de to re-apply
the initial patch. This should only be merged once
I2acfd0b7a1e48aec107ded3bbe4963e2df24f4d3 is deployed.
Change-Id: If12ab65b1d773946fca6c8601ff51290136549c8
This reverts commit ec22840c4a.
This patch currently creates issues on beta, which still runs with the
latest vendor version of Parsoid. If, for some reason, Parsoid doesn't
get deployed with this patch, I2acfd0b7a1e48aec107ded3bbe4963e2df24f4d3
doesn't get included, and the REST page handler breaks.
Staggered deploy seems safer in this context, hence the proposal for a
temporary revert, and a merge after the next Parsoid vendor patch is
deployed.
Change-Id: I3f859fa807a04892a67323cd4e98be0d3fbb1676
* Parsoid's rt-testing script is still a node.js script and hence needs
ucs2 offests for its syntactic / semantic diff classification.
* So, we cannot let 1aa71cf5 ride the train since it will break
Parsoid's rt-testing. We'll figure out an alternative way of handling
it, but for now, I am reverting that part of the patch.
* Document in the ParsoidHandlerTest test that ucs2 offsets are used and
cannot be changed to 'byte'
Bug: T347426
Change-Id: Ifa833e01ef117d7bcd6da1c7eb542535192662eb
The Helper classes are deprecated since 1afd52e3e4.
Depends-On: I2acfd0b7a1e48aec107ded3bbe4963e2df24f4d3
Change-Id: Ie9973c6d6474bb7b4720c0641ca7492dc946d923
* This is in service of a followup patch that merges ParsoidOutputAccess
and ParserOutputAccess. We want to eliminate all Parsoid-specific options
that aren't part of ParserOptions and aren't easily supportable via
html2html transforms.
* offsetType conversion relies on Parsoid code that is a bit entangled
with env, siteconfig (and extension configs), page source, etc. It
could all be refactored but once the html2html output transformation
framework lands, we could potentially use that to call Parsoid to do
these transforms by exposing such transforms to the framework.
* In this patch, outputContentVersion that isn't the default major HTML
version is no longer support. It could potentially be supported via the
downgrade functionality in Parsoid in the future, or we might decide
to re-enable multiple outputContentVersion selection in the future
if such a use case arises. But, there are no plans to bump the major
HTML version in the near future while we work on read views.
* Rather than delete associated tests, I've marked them skipped so that
they can re-enabled when this support is added back.
Bug: T347426
Change-Id: Ibede4acd68e944512f6d00763d29c6b1605d67eb
This class is used heavily basically everywhere, moving it to Utils
wouldn't make much sense. Also with this change, we can move
StatusValue to MediaWiki\Status as well.
Bug: T321882
Depends-On: I5f89ecf27ce1471a74f31c6018806461781213c3
Change-Id: I04c1dcf5129df437589149f0f3e284974d7c98fa
* Explicitly set wrapSections to true. This has have no significant
impact since it defaults to true within Parsoid.
* 'pageName' and 'prefix' removed from ParsoidOutputAccess since
they are not needed / used in Parsoid.
* 'logLinterData' need to be set in the ParserOutputAccess paths.
* A bunch of documentation FIXMEs as I was digging through the code.
* Record a FIXME that ParsoidOutputAccess and ParsoidParser (which
is used in the ParserOutputAccess use page) differ in how they
handle the language value (whether the default value of the title /
page or the pageLanguageOverride from the REST API). ParsoidParser
computes a preferred variant whereas ParsoidOutputAccess right now
does NOT do that. So, as part of the switchover to ParserOutputAccess,
we will need to set disableContentConversion in ParserOptions.
That will happen in a later patch.
Bug: T332931
Change-Id: I7326ae3452a7d496a57f5c4ff2ddeaf0daa7ab70
Avoids any issue with not respecting the explicit version in accept
headers. However, it will effectively mean a cache miss on every
request after a Parsoid version bump.
Bug: T333606
Change-Id: Ia70f819df79fbb12a5b1dd6a98bfe0b968808d18
