This class belongs with the rest of the Parsoid output stash code.
This class has been marked @unstable since 1.39 and thus the move
does not need release notes.
Change-Id: I16061c0c28b1549fbe90ea082cc717fee4a09a6e
Set the render ID for each parse stored into cache so that we are able
to identify a specific parse when there are dependencies (for example
in an edit based on that parse). This is recorded as a property added
to the ParserOutput, not the parent CacheTime interface. Even though
the render ID is /related/ to the CacheTime interface, CacheTime is
also used directly as a parser cache key, and the UUID should not be
part of the lookup key.
In general we are trying to move the location where these cache
properties are set as early as possible, so we check at each location
to ensure we don't overwrite a previously-set value. Eventually we
can convert most of these checks into assertions that the cache
properties have already been set (T350538). The primary location for
setting cache properties is the ContentRenderer.
Moved setting the revision timestamp into ContentRenderer as well, as
it was set along the same code paths. An extra parameter was added to
ContentRenderer::getParserOutput() to support this.
Added merge code to ParserOutput::mergeInternalMetaDataFrom() which
should ensure that cache time, revision, timestamp, and render id are
all set properly when multiple slots are combined together in MCR.
In order to ensure the render ID is set on all codepaths we needed to
plumb the GlobalIdGenerator service into ContentRenderer, ParserCache,
ParserCacheFactory, and RevisionOutputCache. Eventually (T350538) it
should only be necessary in the ContentRenderer.
Bug: T350538
Bug: T349868
Followup-To: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
Change-Id: I72c5e6f86b7f081ab5ce7a56f5365d2f75067a78
Some less trivial cases. Also update variable names.
This doesn't really change anything, but I hope it helps promote
getConnectionProvider() as the common way to do this.
Follow-up to 8604c384f6.
Change-Id: I6657d783375fac5c7fa856b884ff1fb09285e94c
Changes to the use statements done automatically via script
Addition of missing use statements and changes to docs done manually
Change-Id: Ib326ae1e5c8409a98398c721e8b8ce42c73bd012
This was done automatically using the
`Universal.WhiteSpace.CommaSpacing` sniff, which will be included in the
next release of the MW PHPCS config.
Some of these have been adjusted manually where the autofix broke
vertical alignment.
Change-Id: I54a4668d8a2759b9d7de47742c943a535a04e211
For example:
* assertStatusGood = ok without any errors or warnings
* assertStatusWarning = ok, but not good, i.e. there is a warning
Change-Id: I4b3ec7a3c5b028c0505e1371c297a9c47e448b42
While we can wrap these in an additional `fn ()` we can also use
"lcfirst" directly. Function names encoded as strings are valid
callables in PHP.
Change-Id: I966046d5e357b409c57a9bff8a4a5737c1e22b03
* Updated ParserOutput to set Parsoid render ids that REST API
functionality expects in ParserOutput objects.
* CacheThresholdTime functionality no longer exists since it was
implemented in ParsoidOutputAccess and ParserOutputAccess doesn't
support it. This is tracked in T346765.
* Enforce the constraint that uncacheable parses are only for fake or
mutable revisions. Updated tests that violated this constraint to
use 'getParseOutput' instead of calling the parse method directly.
* Had to make some changes in ParsoidParser around use of preferredVariant
passed to Parsoid. I also left some TODO comments for future fixes.
T267067 is also relevant here.
PARSOID-SPECIFIC OPTIONS:
* logLinterData: linter data is always logged by default -- removed
support to disable it. Linter extension handles stale lints properly
and it is better to let it handle it rather than add special cases
to the API.
* offsetType: Moved this support to ParsoidHandler as a post-processing
of byte-offset output. This eliminates the need to support this
Parsoid-specific options in the ContentHandler hierarchies.
* body_only / wrapSections: Handled this in HtmlOutputRendererHelper
as a post-processing of regular output by removing sections and
returning the body content only. This does result in some useless
section-wrapping work with Parsoid, but the simplification is probably
worth it. If in the future, we support Parsoid-specific options in
the ContentHandler hierarchy, we could re-introduce this. But, in any
case, this "fragment" flavor options is likely to get moved out of
core into the VisualEditor extension code.
DEPLOYMENT:
* This patch changes the cache key by setting the useParsoid option
in ParserOptions. The parent patch handles this to ensure we don't
encounter a cold cache on deploy.
TESTS:
* Updated tests and mocks to reflect new reality.
* Do we need any new tests?
Bug: T332931
Change-Id: Ic9b7cc0fcf365e772b7d080d76a065e3fd585f80
The use of Article::getRedirectHeaderHtml() has been discouraged for a
while, since WikitextContentHandler can (should) be used to insert the
redirect header. Further, since I20db09619999919bfeda997d79561d21e3bf8718
the header should be added as an extension data property instead of
directly concatenated to the HTML. Regardless, this functionality
logically should live in LinkRenderer.
Change-Id: I4d0de0e72473ae039dca420a2733bc746d8c2951
The method should never be called directly, so make it throw an exception.
Nonetheless, mark it as deprecated and detect overrides in the
constructor, so that anyone who tries to override this method will see a
warning.
Fix the few tests that were relying on the existence of the test page.
Bug: T342428
Depends-On: Ic64ded5e2c0b59e7c888ece9566076058a125be4
Change-Id: I308617427309815062d54c14f3438cab31b08a73
This class is used heavily basically everywhere, moving it to Utils
wouldn't make much sense. Also with this change, we can move
StatusValue to MediaWiki\Status as well.
Bug: T321882
Depends-On: I5f89ecf27ce1471a74f31c6018806461781213c3
Change-Id: I04c1dcf5129df437589149f0f3e284974d7c98fa
* Add MediaWikiIntegrationTestCase::newSelectQueryBuilder(), which
creates a subclass of SelectQueryBuilder with assert methods.
* Migrate most callers of assertSelect() to this new query builder
interface.
Bug: T311866
Change-Id: I7392b37988067020d5f684276320dae0a474631a
Follows-up I4c7d826c7ec654b, I1287f3979aba1bf1.
We lose useful coverage and spend valuable time keeping these accurate
through refactors (or worse, forget to do so). The theoretically "bad"
accidental coverage is almost never actually bad.
Having said that, I'm not removing them wholesale (yet). I've audited
each of these specific files to confirm it is a general test of the
specified subject class, and also kept it limited to those specified
classes. That's imho more than 100% of the benefit for less than 1%
of the cost (more because `@covers` is more valuable than the fragile
and corrosive individual private method tracking in tests that
inevitably get out of date with no local incentive to keep them up to
date).
Cases like structure tests keep `@coversNothing` etc and we still don't
count coverage of other classes. There may be a handful of large
legacy classes where some methods are effectively class-like in
complexity and that's why it's good for PHPUnit to offer the precision
instrument but that doesn't meant we have to use that by-default for
everything.
I think best practice is to write good narrow unit tests, that reflect
how the code should be used in practice. Not to write bad tests and
hide part of its coverage within the same class or even namespace.
Fortunately, that's generally what we do already it's just that we
also kept these annotations still in many cases.
This wastes time to keep methods in sync, time to realize (and fix)
when other people inevitably didn't keep them in sync, time to find
uncovered code only to realize it is already covered, time for a less
experienced engineer to feel obligate to and do write a low quality
test to cover the "missing" branch in an unrealistic way, time wasted
in on-boarding by using such "bad" tests as example for how to use
the code and then having to unlearn it months/years later, loss of
telemetry in knowing what code actually isn't propertly tested due to
being masked by a bad test, and lost oppertunities to find actually
ununused/unreachable code and to think about how to instead structure
the code such that maybe that code can be removed.
------
Especially cases like LBFactoryTest.php were getting out of hand,
and in GlobalIdGeneratorTest.php we even resorted to reminding people
with inline comments to keep tags in sync.
Change-Id: I69b5385868cc6b451e5f2ebec9539694968bf58c
This is an initial quick-and-dirty implementation. The
ParsoidParser class will eventually inherit from \Parser,
but this is an initial placeholder to unblock other Parsoid
read views work.
Currently Parsoid does not fully implement all the ParserOutput
metadata set by the legacy parser, but we're working on it.
This patch also addresses T300325 by ensuring the the Page HTML
APIs use ParserOutput::getRawText(), which will return the entire
Parsoid HTML document without post-processing. This is what
the Parsoid team refers to as "edit mode" HTML. The
ParserOutput::getText() method returns only the <body> contents
of the HTML, and applies several transformations, including
inserting Table of Contents and style deduplication; this is
the "read views" flavor of the Parsoid HTML.
We need to be careful of the interaction of the `useParsoid` flag with
the ParserCacheMetadata. Effectively `useParsoid` should *always* be
marked as "used" or else the ParserCache will assume its value doesn't
matter and will serve legacy content for parsoid requests and
vice-versa. T330677 is a follow up to address this more thoroughly by
splitting the parser cache in ParserOutputAccess; the stop gap in this
patch is fragile and, because it doesn't fork the ParserCacheMetadata
cache, may corrupt the ParserCacheMetadata in the case when Parsoid
and the legacy parser consult different sets of options to render a
page.
Bug: T300191
Bug: T330677
Bug: T300325
Change-Id: Ica09a4284c00d7917f8b6249e946232b2fb38011
Just methods where adding "static" to the declaration was enough, I
didn't do anything with providers that used $this.
Initially by search and replace. There were many mistakes which I
found mostly by running the PHPStorm inspection which searches for
$this usage in a static method. Later I used the PHPStorm "make static"
action which avoids the more obvious mistakes.
Bug: T332865
Change-Id: I47ed6692945607dfa5c139d42edbd934fa4f3a36
testDoParserCacheUpdate was using a mix of real revisions and fake
revisions. This was getting confusing, since synchronous updates would
use the fakes, while updates that use the job queue were seeing only the
real revisions in the database.
Change-Id: I4178cefd116ae105f7c6a15dc4cbe4693309b1e7
This patch introduces ParsoidParserCachePrewarm job
that is used to warm PC with parsoid outputs in order
to speed up page reads on large wikis.
Bug: T322427
Change-Id: Ib63a02d3cf5348b36f4f166ff6939f4d2e7fef6f
It misrepresents the users contribution to show empty text for a
revision when in fact the revision contained some text which we later
lost.
Also, errors from SqlBlobStore::fetchBlobs() did not stop a cache entry
from being written, so a subsequent cache hit would show the bad
revision as empty.
So, in Storage:
* Add BadBlobException, which is thrown by the Storage layer to
indicate that a revision is marked as bad.
* Have SqlBlobStore::getBlobStore() return an error for bad blobs
instead of an empty string.
* Duplicate the check for flags=error into SqlBlobStore::expandBlob().
This avoids an unnecessary cache fetch, and avoids making
decompressData() throw on error, which would be a b/c break.
* In SqlBlobStore::getBlob(), suppress the cache when there was an
error.
In Revision:
* Add BadRevisionException, to wrap BadBlobException in the Revision
layer.
* Return null from RevisionRecord::getContent() on a broader set of
errors. Make it mostly non-throwing.
* Add RevisionRecord::getContentOrThrow() which returns a non-nullable
Content.
* Note that SlotRecord::getContent() returns a non-nullable Content so
now throws in more cases.
In the UI:
* In Article::view(), catch the exception and show an error message.
* In DifferenceEngine, catch the exception and make a suitable error
message available via getRevisionLoadErrors(). In the diff page, show
the error message in a box.
* In ApiComparePages and the legacy rvdiffto, show a warning.
* In RawAction, show a 404 by analogy with other error cases.
* In EditPage, there was already handling for $content=null with an
appropriate error message (missing-revision-content). But having
$this->textbox1 = null caused PHP 8.1 deprecation warnings, so I fixed
that.
* In EditPage undo, there was already handling for null content, but I
improved the error message: "does not exist or was deleted" seems more
appropriate than "conflicting intermediate edits".
Change-Id: Idd1278d6d756ef37d64addb7b5f3be30747ea603
* Add a safe unserialize() wrapper for HistoryBlob classes
* Add a safe unserialize() wrapper for plain array data as used for
compressed internal storage by ConcatenatedGzipHistoryBlob and
DiffHistoryBlob.
* Fix tests broken by this.
* Fix unnecessary call to uncompress(), __wakeup() does this already.
Was a phan error now that we have more information about the type of
$obj.
* Add tests for successful unserialize and wakeup of WMF production
data.
Change-Id: Ic995dda16d9c6045b33f2fdae7f6575ac8329976
PHPUnit 3.7.0 was the first version to support @requires extension. At
the time checkPHPExtension() was added in a7901801b4, MediaWiki
still supported PHPUnit 3.6.7.
MediaWiki now requires intl and xml, so I removed checks for those
extensions rather than converting them to annotations.
checkPHPExtension() is removed without deprecation; it does not appear
to have ever been used (and is not likely to be used) in MW extensions.
This is explicitly permitted under the stable interface policy. Even if
it were not, only tests are affected, and they are supposed to fail
anyway if hard deprecated code is used.
Change-Id: I45f9b4c0e120683103cead916f4d4ef58bd11530
