Commit graph

29 commits

Author SHA1 Message Date
Adam Wight
b0b7723977 Cleanup: const access
Change-Id: Iedfbd09d90024c9876e8d58b510f5775c3028520
2019-12-14 14:24:36 +01:00
daniel
e1f8a8148d SqlBlobStore HOT FIX: remove caching from getBlobBatch
This is a TEMPORARY HACK to fix a problem with cached values
getting mixed up when using getBlobBatch(). As of the creation of this
patch, the actual cause is unknown. Caching should be put back once the
cause of the problem has been found.

Note that getBlobBatch is only used in maintenance scripts.
Removal of caching in that method has no effect on web requests.

getBlobBatch() was introduced in I56306c50a661 and is used by
RevisionStore::getContentBlobsForBatch(), which in turn is needed
by the Translate extension to remove its dependency on the deprecated
rev_text_id field.

Bug: T235188
Change-Id: I94c6f9ba7b9caeebaa9b055916f15f7bbdcd8fb6
2019-10-16 23:02:52 +00:00
Timo Tijhof
01095f8075 Storage: SqlBlobStore no longer needs Lanugage object
Constructing a Language object in order to initialize the
BlobStoreFactory service causes a circular dependency
(see T231866).

SqlBlobStore was using the Language object to all iconv.
But nothing language specific is done in Language::iconv,
so we can just inline the call.

Bug: T231866
Change-Id: I90c25decbcff10ea762a2c7474a12fd2041b3abc
2019-10-02 09:26:34 -07:00
Timo Tijhof
fe3c418925 Storage: Use a more descriptive WAN cache keyclass in 'SqlBlobStore'
In order to make statistics from WANObjectCache more useful,
each kind of key should generally have a unique first make*Key
component. This usually corresponds 1:1 to a call to make*Key,
unless the same key format is duplicated in muliple places.

In this case, change ('BlobStore', 'address', …, …)
to ('SqlBlobStore-blob', …, …).

Bug: T197849
Change-Id: Iba3ae1bf947d6f56c72df2cf3f2160d458d38c66
2019-09-14 02:22:01 +00:00
Petr Pchelko
fa36773e58 Introduce BlobStore::getBlobBatch method
Bug: T230834
Change-Id: I56306c50a6617dc91e4eb362ef010703ac25d951
2019-08-30 09:40:49 +00:00
Aaron Schulz
83e294a892 Convert LoadBalancer::getConnection() callers to LoadBalancer::getConnectionRef()
Change-Id: Ie1ae322202105ce093779e6d0b00f335c71be31f
2019-07-21 01:02:49 -07:00
Aaron Schulz
baafb5adb4 Make ExternalStore wrap ExternalStoreFactory and create access class
* Inject settings and global instances as dependencies to the
  ExternalStoreMedium instances. This includes the local wiki
  domain, so that wfWikiId() calls are not scattered around.
* Create ExternalStoreAccess service for read/write logic.
* Deprecate the ExternalStore wrapper methods.
* Add some exception cases for bogus store URLs are used instead
  of just giving PHP warnings and failing later.
* Make moveToExternal.php require the type/protocol to decide
  which ExternalStoreMedium to use instead of assuming "DB".
* Convert logging calls to use LoggerInterface.

Change-Id: I40c3b5534fc8a31116c4c5eb64ee6e4903a6197a
2019-06-28 14:31:44 -07:00
Aaron Schulz
a9a13a1266 storage: rename various $wikiId fields/parameters to $dbDomain
These fields are passed to methods like LoadBalancer::getConnection() and are
already expected to be DB domains. Update various comments as well.

Fix a few minor IDEA warnings.

Change-Id: If480cc4d7d4a3c2d176ab346e6307e80cd2540c5
2019-06-28 00:57:04 +00:00
jenkins-bot
7b3631abc8 Merge "Storage: Type against ILBFactory and ILoadBalancer in storage classes" 2019-06-14 00:05:03 +00:00
Umherirrender
29cc0aa0f9 Storage: Type against ILBFactory and ILoadBalancer in storage classes
Instead of the LBFactory/LoadBalancer implementations.

Change-Id: Ia58d9056bd60f3f094fbb16f7b9afa64e11143c8
2019-06-13 22:10:58 +00:00
Daimona Eaytoy
f5eaf6470b externalstore: Fix insert*() return docs and remove redundant checks
ExternalStorage::insertWithFallback is reported to return false on
failure, but it doesn't. It has a single exit point, and return value is
checked with strlen(), so actually it can only return the URL or throw.
Thus, update any related doc and remove a redundant check from code
calling insertToDefault.

Change-Id: Ic95c3aed19118b987aef105f8077d55558f39127
2019-06-13 18:23:23 +00:00
jenkins-bot
68b12dfded Merge "Make BackupDumper MCR compatible (main slot only)" 2019-03-20 02:15:43 +00:00
daniel
5988e35505 Make BackupDumper MCR compatible (main slot only)
This makes BackupDumper compatible with the new mechanism for accessing
revision content.

This requires some changes to the way database connections are re-used,
since RevisionStore/SqlBlobStore needs to be able to run queries against
the database while the overall result set is being streamed.

This change does not yet add handing for extra slots to BackupDumper.
That first needs a spec for how extra slots will be represented in the
XML schma (T174031).

NOTE: this changes the output of fetchText from using integer text_id
values to using content_address values (e.g. "tt:4567" for text row
with old_id 4567). It also changes fetchText to accept such addresses
as input, for forward-compatibility. XML stub dumps still use the
numeric format in the id attribute, pending T199121.

Bug: T198706
Change-Id: If4c31b7975b4d901afa8c194c10446c99e27eadf
2019-03-14 13:19:51 +00:00
Aaron Schulz
5240014044 Do not bother with WAN cache set options for immutable text blob keys
Doing so can only result in occasional needless set() call rejections
or automatically lowered TTLs.

Change-Id: I2f007ba88e8f9c9f1ac3cd3d2daf3cd2289e315a
2019-03-06 22:33:04 +00:00
daniel
63ae40bfad Document cross-wiki cache sharing logic.
This makes explicit the fact that some classes can use the local wiki's default
WANObjectCache instance while still sharing cache entries between wikis.

Bug: T198704
Change-Id: Ic29f79e4b4bd6bdaa844049962a163e72a59c45a
2018-08-02 18:55:41 +00:00
daniel
53fd8295ff Use consistent caching strategy in Revision storage classes
DEPLOYMENT: This changes the cache key for revision
content blobs. Expect a brief rise in ExternalStore hits.

Bug: T198704
Change-Id: Icc2d16bc5a1e27ba4caea49a784ba7aeac15042a
2018-07-05 17:44:24 +02:00
jenkins-bot
9d914bed08 Merge "Blob can't be false" 2018-06-20 20:04:10 +00:00
daniel
564257dc50 Blob can't be false
Change-Id: Ic06fcfaac71128c0ff7e9079685eac18206f2004
2018-06-20 17:32:35 +02:00
daniel
2f9e892c74 When encountering bad blobs, log the text row id.
Change-Id: Ie7c932f229854fd3582d507fa7cc154ffdd21f55
2018-06-19 13:59:01 +02:00
Umherirrender
7eb7f04771 Change @throw to @throws
Phpcs does not look at private functions

Change-Id: I9adedf1f1c488e74db7de89e97cb6a481bf7d567
2018-04-27 21:53:19 +02:00
daniel
1b115fd022 Introduce SqlBlobStore::makeAddressFromTextId
This introduces a convenience method for constructing a blob address
from a text ID. It's the inverse of SqlBlobStore::getTextIdFromAddress

Change-Id: I31b3ee5e40185c754fb2c119eb5edc50b903f5dc
2018-04-19 17:23:59 +00:00
Aaron Schulz
d9ba7cd005 Make LocalFile check early if the revision store is available
This reduces the odds of having files without corresponding
wiki pages, given that the later is done in a deferred update.

Also made some documentation cleanups.

Bug: T187942
Change-Id: Iff516669f535713d37e0011e2d7ed285c667f1c5
2018-02-22 22:07:30 +00:00
Brad Jorsch
a7c103f24d RevisionStore: Remove reference to unimplemented 'ar' blob address schema
It was thought that an 'ar:' schema for SqlBlobStore addresses would be
needed to support archive rows from before MediaWiki 1.5 that still
stored the text in ar_text and ar_flags instead of using the text table
and ar_text_id. But this schema was never actually implemented.

Now I5608c6b6 has migrated that content to the text table and
ar_text_id, and I18f1c740b will remove the database fields entirely. So
there's no reason to ever implement this schema.

Change-Id: Ic449478c244bb13a8c5139e20488e876c0a793bb
2018-02-22 10:44:21 -05:00
daniel
cb94d35c79 Document expandBlob behavior when no flags are given.
Bug: T184749
Change-Id: I5f1f029d928a7bc25877b0eae9f3822ec321b24a
2018-01-14 08:59:42 +00:00
Aaron Schulz
2e90693ca7 Use cache set options in SqlBlobStore::getBlob
This is useful for sanity in the case of DB rollback.

Change-Id: I2e50183835a97bc82efee53a407a624f0e35f53f
2018-01-12 18:42:16 +00:00
Brad Jorsch
295b055ea3 Migrate ar_text to modern storage
This really should have been done a very long time ago.

This adds a maintenance script to migrate rows with ar_text_id null to
modern storage, either the text table or ExternalStore.

Bug: T36925
Change-Id: I5608c6b6d3ecad516b785e13d668427c1b762e41
2018-01-09 16:15:44 -05:00
addshore
e51f95dea3 [MCR] Introduce BlobStoreFactory
This allows Revision::getRevisionText to get
a different BlobStore instance when $wiki is passed in
restoring the behaviour for $wiki before the MCR Revision
overhaul patch was merged.
Ia4c20a91e98df0b9b14b138eb4825c55e5200384

Bug: T183634
Bug: T183631
bug: T183583
Change-Id: Ib0949454e9a003c2965adc1aab38e31fcf121afe
2017-12-24 23:22:30 +00:00
addshore
a10315f2fc [MCR] Fix SqlBlobStore using DB_REPLICA for writes
Bug: T183242
Bug: T183245
Bug: T183252
Change-Id: Ic0b9910b5de5ce6817d6a4a3e5e32103e2113926
2017-12-19 14:58:06 +00:00
daniel
e61a1caadd [MCR] Break Revision into RevisionRecord and RevisionStore
Bug: T174025
Change-Id: I140f43a6fb443b38483f41f268c906b9cea64cf7
2017-12-13 22:34:29 +00:00