When encountering missing or null text blobs, populateContentTables
would fail, leaving the data migration to slots incomplete. This
patch allows the script to continue, reporting the bad blobs without
failing.
This patch and I041d5c67bf241bb7cf2e31e0a4b0030e2cf9a13b, upon
which it depends upon, should be backported.
Bug: T212428
Change-Id: I582eac2befec152cb48d898644bc024a5ade2260
Updates for the removal of the Revision class itself
and the various methods/hooks/variables removed in the
process, including:
- Update some documentation removing most references
to the Revision class and updating the MCR migration
notes to reflect the past tense for Revision methods.
- Change some capitalization from "Revision" to "revision"
to make it clear comments are about revisions in general,
not the Revision class in particular.
- Minor code tweaks including removing unused variables that
were around for the old hooks that were removed, and
removing the use of DeprecatablePropertyArray where no
longer needed for anything.
- Fix incorrect documentation for PageUpdater::getStatus(),
the status value changed a while ago to have revision-record
in addition to revision, and recently to only have the
revision-record, but ironically PageUpdater was never updated.
- Removed Parser::$mRevisionObject, used to be a Revision object
and was deprecated in 1.35, missed earlier because it was no
longer being set to Revision objects, always null.
- Add RevisionRecord typehints in DummyLinker to match those
in the corresponding Linker methods
This should be a no-op in terms of functionality.
Bug: T247143
Change-Id: I03bbb94fc29085855448780b1a5ad9063911ecc4
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.
Created by I25a17fb22b6b669e817317a0f45051ae9c608208
Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
Advertize the fact that production errors resulting from corruption in
the content storage layer can be remedied using findBadBlobs.php.
Change-Id: I4eab773d0d2da57a545f211c2fef36146331c570
A terminating line break has not been required in wfDebug() since 2014,
however no migration was done. Some of these line breaks found their way
into LoggerInterface::debug() calls, where they mess up the formatting
of the debug log.
So, remove terminating line breaks from wfDebug() and
LoggerInterface::debug() calls.
Also:
* Fix the stripping of leading line breaks from the log header emitted
by Setup.php. This feature, accidentally broken in 2014, allows
requests to be distinguished in the log file.
* Avoid using the global variable $self.
* Move the logging of the client IP back to Setup.php. It was moved to
WebRequest in the hopes that it would not always be needed, however
$wgRequest->getIP() is now called unconditionally a few lines up in
Setup.php. This means that it is put in its proper place after the
"start request" message.
* Wrap the log header code in a closure so that variables like $name do
not leak into global scope.
* In Linker.php, remove a few instances of an unnecessary second
parameter to wfDebug().
Change-Id: I96651d3044a95b9d210b51cb8368edc76bebbb9e
This script scans for content blobs that can't be loaded due to
database corruption, and can change their entry in the content table
to an address starting with "bad:". Such addresses cause the content
to be read as empty, with no log entry. This is useful to avoid
errors and log spam due to known bad revisions.
The script is designed to scan a limited number of revisions from a
given start date. The assumption is that database corruption is
generally caused by an intermedia bug or system failure which will
affect many revisions over a short period of time.
Bug: T205936
Change-Id: I6f513133e90701bee89d63efa618afc3f91c2d2b
This adds support for addresses starting with the prefix "bad:",
which will always resolve to an empty string.
The syntax for addresses supported by SqlBlobStore is relaxed
to be closer to RFC 3986 and make it easier to include meaningful
information after the "bad:" prefix.
Bug: T240686
Change-Id: I175b1880ce94ff8a9079bc50440cde188e57d95d
This is a TEMPORARY HACK to fix a problem with cached values
getting mixed up when using getBlobBatch(). As of the creation of this
patch, the actual cause is unknown. Caching should be put back once the
cause of the problem has been found.
Note that getBlobBatch is only used in maintenance scripts.
Removal of caching in that method has no effect on web requests.
getBlobBatch() was introduced in I56306c50a661 and is used by
RevisionStore::getContentBlobsForBatch(), which in turn is needed
by the Translate extension to remove its dependency on the deprecated
rev_text_id field.
Bug: T235188
Change-Id: I94c6f9ba7b9caeebaa9b055916f15f7bbdcd8fb6
Constructing a Language object in order to initialize the
BlobStoreFactory service causes a circular dependency
(see T231866).
SqlBlobStore was using the Language object to all iconv.
But nothing language specific is done in Language::iconv,
so we can just inline the call.
Bug: T231866
Change-Id: I90c25decbcff10ea762a2c7474a12fd2041b3abc
In order to make statistics from WANObjectCache more useful,
each kind of key should generally have a unique first make*Key
component. This usually corresponds 1:1 to a call to make*Key,
unless the same key format is duplicated in muliple places.
In this case, change ('BlobStore', 'address', …, …)
to ('SqlBlobStore-blob', …, …).
Bug: T197849
Change-Id: Iba3ae1bf947d6f56c72df2cf3f2160d458d38c66
* Inject settings and global instances as dependencies to the
ExternalStoreMedium instances. This includes the local wiki
domain, so that wfWikiId() calls are not scattered around.
* Create ExternalStoreAccess service for read/write logic.
* Deprecate the ExternalStore wrapper methods.
* Add some exception cases for bogus store URLs are used instead
of just giving PHP warnings and failing later.
* Make moveToExternal.php require the type/protocol to decide
which ExternalStoreMedium to use instead of assuming "DB".
* Convert logging calls to use LoggerInterface.
Change-Id: I40c3b5534fc8a31116c4c5eb64ee6e4903a6197a
These fields are passed to methods like LoadBalancer::getConnection() and are
already expected to be DB domains. Update various comments as well.
Fix a few minor IDEA warnings.
Change-Id: If480cc4d7d4a3c2d176ab346e6307e80cd2540c5
ExternalStorage::insertWithFallback is reported to return false on
failure, but it doesn't. It has a single exit point, and return value is
checked with strlen(), so actually it can only return the URL or throw.
Thus, update any related doc and remove a redundant check from code
calling insertToDefault.
Change-Id: Ic95c3aed19118b987aef105f8077d55558f39127
This makes BackupDumper compatible with the new mechanism for accessing
revision content.
This requires some changes to the way database connections are re-used,
since RevisionStore/SqlBlobStore needs to be able to run queries against
the database while the overall result set is being streamed.
This change does not yet add handing for extra slots to BackupDumper.
That first needs a spec for how extra slots will be represented in the
XML schma (T174031).
NOTE: this changes the output of fetchText from using integer text_id
values to using content_address values (e.g. "tt:4567" for text row
with old_id 4567). It also changes fetchText to accept such addresses
as input, for forward-compatibility. XML stub dumps still use the
numeric format in the id attribute, pending T199121.
Bug: T198706
Change-Id: If4c31b7975b4d901afa8c194c10446c99e27eadf
Doing so can only result in occasional needless set() call rejections
or automatically lowered TTLs.
Change-Id: I2f007ba88e8f9c9f1ac3cd3d2daf3cd2289e315a
This makes explicit the fact that some classes can use the local wiki's default
WANObjectCache instance while still sharing cache entries between wikis.
Bug: T198704
Change-Id: Ic29f79e4b4bd6bdaa844049962a163e72a59c45a
DEPLOYMENT: This changes the cache key for revision
content blobs. Expect a brief rise in ExternalStore hits.
Bug: T198704
Change-Id: Icc2d16bc5a1e27ba4caea49a784ba7aeac15042a
This introduces a convenience method for constructing a blob address
from a text ID. It's the inverse of SqlBlobStore::getTextIdFromAddress
Change-Id: I31b3ee5e40185c754fb2c119eb5edc50b903f5dc
This reduces the odds of having files without corresponding
wiki pages, given that the later is done in a deferred update.
Also made some documentation cleanups.
Bug: T187942
Change-Id: Iff516669f535713d37e0011e2d7ed285c667f1c5
It was thought that an 'ar:' schema for SqlBlobStore addresses would be
needed to support archive rows from before MediaWiki 1.5 that still
stored the text in ar_text and ar_flags instead of using the text table
and ar_text_id. But this schema was never actually implemented.
Now I5608c6b6 has migrated that content to the text table and
ar_text_id, and I18f1c740b will remove the database fields entirely. So
there's no reason to ever implement this schema.
Change-Id: Ic449478c244bb13a8c5139e20488e876c0a793bb
This really should have been done a very long time ago.
This adds a maintenance script to migrate rows with ar_text_id null to
modern storage, either the text table or ExternalStore.
Bug: T36925
Change-Id: I5608c6b6d3ecad516b785e13d668427c1b762e41
This allows Revision::getRevisionText to get
a different BlobStore instance when $wiki is passed in
restoring the behaviour for $wiki before the MCR Revision
overhaul patch was merged.
Ia4c20a91e98df0b9b14b138eb4825c55e5200384
Bug: T183634
Bug: T183631
bug: T183583
Change-Id: Ib0949454e9a003c2965adc1aab38e31fcf121afe
