Commit graph

59 commits

Author SHA1 Message Date
James D. Forrester
597376e12c Replace getMasterDB methods with getPrimaryDB
Bug: T254646
Change-Id: I68198bc39b174ea1920b4acc2617cb6c6ce406e9
2021-07-14 16:21:04 -07:00
Tim Starling
68ebdfc77f Optionally split out parts of file metadata to BlobStore
* Optionally store metadata in the database in JSON format instead of
  PHP serialization. The new JSON format has a top-level "envelope"
  array which gives us a place to store things that are not part of the
  handler metadata.
* Optionally split metadata items, putting items above a threshold into
  the text table. The FileRepo and MediaHandler must both opt in.
* For staged deployment, the read side of these changes is always
  active. Only the write side is configurable.

Bug: T275268
Change-Id: I876ea5c9d3a1881e278f689d2f8a3ae20240c703
2021-06-11 08:01:26 +10:00
James D. Forrester
df5eb22f83 Replace uses of DB_MASTER with DB_PRIMARY
Just an auto-replace from codesniffer for now.

Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
2021-04-29 09:24:31 -07:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Aaron Schulz
70c4255978 filerepo: clean up shared cache keys to avoid key metrics clutter
This avoids statsd metrics with names that begin with DB domains

Bug: T261534
Change-Id: I69fa4482c639bd49bdc2dec48a8dcdd74d455432
2020-10-21 13:29:15 -07:00
Aaron Schulz
58c94afe2e filerepo: make ForeignDBRepo::getSharedKey() use the DB domain ID for consistency
Follows-up 96ad0db6b4 and 88e17d3f7c.

Change-Id: I5e3163e54006b35792a82264905c637b187d25dd
2020-05-25 20:54:39 +00:00
Aaron Schulz
96ad0db6b4 filerepo: use makeGlobalKey() in ForiegnDBViaLBRepo::getSharedCacheKey()
Keys used by multiple wikis should use this key generation method.
It also means settings like "coalesceKeys" in WANObjectCache will
correctly identify these keys as being global.

This change should be deployed on all wikis at once.

Bug: T253405
Change-Id: Ie19613ef7643af92aeb56db203e3d79139143231
2020-05-22 21:24:37 +00:00
Reedy
af063dd794 Fix more Squiz.Scope.MethodScope.Missing
Change-Id: I44cd7ba39a898a27f0f66cf34238ab95370d2279
2020-05-18 21:02:14 +00:00
Aaron Schulz
0d933c6760 Deprecate wfForeignMemcKey() and remove the last call
Bug: T250407 
Change-Id: I82ca5a8a5d04166263eb19c4c99d72321bbbcdec
2020-04-17 09:57:37 +00:00
Aaron Schulz
2da2011979 filerepo: clean up File::getRepo() subclass return type comments
Change-Id: I7fbe19c96cc41e9f4534e56853fd119b74b2603c
2020-03-19 16:40:15 -07:00
Daimona Eaytoy
d3252fc870 Fix new phan errors, part 5
Some of the errors are suppressed because they're phan false positives.
The idea behind this is that they'll be fixed in a future version of
phan, and we'll just have to remove the suppressions.

Note: I'm disabling UnusedPluginSuppression so that we can start suppressing
issues even if they're still disabled. The sniff should be re-enabled
as soon as we upgrade phan.

Bug: T231636
Change-Id: I0f7fa06a9e03fbb86c7a5eb6e50a850bb258a7f7
2019-10-18 20:45:53 +00:00
Daimona Eaytoy
69cadf44a9 Unsuppress PhanParamsTooMany
This is the last repo-wide suppressed issue. Hurrah.

Bug: T231636
Change-Id: I3dc939f115bea14848c9c40bc52a7601f86ca0a7
2019-10-10 12:44:08 -07:00
Aaron Schulz
8f9d373adc filerepo: remove pointless "foreign" flag in ForeignDBRepo::getMasterDB()
This connection handle is not managed by any LoadBalancer, so nothing checks that

Change-Id: Ie60859aacbcccd8439bc58d816c6b94769a4331f
2019-06-28 17:33:50 -07:00
Max Semenik
8085c58a4b Replace call_user_func_array(), part 1
Uses new PHP 5.6 syntax like ...parameter unpacking and
calling anything looking like a callback to make the code more readable.
There are much more occurrences but this commit is intentionally limited
to an easily reviewable size.

Change-Id: Idcec077ef3fdf029b632cceafd0150851ad723e3
2018-06-04 23:39:04 -07:00
Umherirrender
3124a990a2 Use ::class to resolve class names in includes files
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I07a925c2a9404b0865e8a8703864ded9d14aa769
2018-01-27 20:34:29 +01:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Timo Tijhof
0c5b6ecc8f Clean up get_class() in /includes/filerepo and /includes/resourceloader
* get_class()        -> __CLASS__ (same as self::class)
* get_called_class() -> static::class
* get_class($this)   -> static::class

Change-Id: Ic4f9b150ddc35e25f51297de7fc12410d421be4d
2017-03-07 21:30:29 +00:00
Aaron Schulz
f525c72590 Rename getSlaveDB() FileRepo method to getReplicaDB()
The old name is left as an alias.

Change-Id: I60ab2cd5ce05df4247d5e25b017d2debee56554e
2016-11-18 07:42:39 -08:00
Aaron Schulz
8585cc9ffb Fix numerous FileRepo IDEA warnings
Change-Id: I3522f37b675efffb68dec9d125faacd8c8776e64
2016-10-05 16:38:15 -07:00
Aaron Schulz
5bbac35ed1 Replace DatabaseBase:: with Database:: and update type hints
Change-Id: I3919b04eb2de4fa0bf8a02239fb5bbf17d347511
2016-09-27 04:20:03 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
44958ebd11 Make ForeignDBRepo::getDBFactory() actually work on 5.3
Change-Id: Id790ccb1728a6a163f2e2149dd56ec1b14fc33af
2015-10-08 07:42:38 +00:00
Aaron Schulz
7c07943eab Update more docs and type hints to use IDatabase
Change-Id: I8c8d85b32a8aba21e14d2a2dde4c25eb509186c1
2015-10-06 18:49:52 -07:00
Aaron Schulz
f1e1313a16 Support for storing files under SHA-1 names
* Added a "storageLayout" flag to LocalRepo config (supports "sha1")
* Added a simple migration script to copy files the SHA1 paths
* Currently works with img_auth.php + thumb_handler.php for URLs
* Added visibility to some LocalFile methods
* Simple tests for the wrapper class.

Co-Authored-By: Gilles Dubuc <gdubuc@wikimedia.org>
Change-Id: Iad46ad669c8ae3c02d10da10c3f7a16fe161663f
Bug: T1210
2015-07-21 16:19:22 +02:00
Gergő Tisza
8073d4f226 Propagate the favicon information to getInfo() for local repos
This is needed by MultimediaViewer to show a relevant icon next
to the local and foreign db repo link in the description area

The setting ends up in configuration for foreign DB, because the
favicon value is stored in the configuration of a wiki, not in its
database.

Change-Id: I3d0599c1afa13e1d8540ff3c8bc34129a55792c5
2014-01-17 11:31:16 +01:00
Siebrand Mazeland
417d4fbbfa Update documentation for file repo related classes
Change-Id: I626d972f92934afc9349e11d3a5550c4044dcb53
2013-12-04 23:06:53 +01:00
Siebrand Mazeland
c3bef4243e Declare visibility on class properties of file repo classes
Change-Id: Ia608df217bb893dddae3219b3315cb44e82f5092
2013-11-25 06:40:42 +01:00
Siebrand Mazeland
bb39b5a6e4 Update formatting of file repo classes
Change-Id: I780070e07352d84d7cda7628994b28b04e8a1ebf
2013-11-23 21:00:11 +01:00
Liangent
af154bea2b Never prefix table names with $wgSharedDB when used in foreign DB
Now database classes have an extra "foreign" property when used in
foreign databases, where $wgSharedDB and $wgSharedTables are not
considered.

Bug: 48819
Change-Id: I45db6860d0a4b4480e0b93aa7e750f096424e40f
2013-05-27 04:52:20 +00:00
umherirrender
1bfc8feb25 Fixed spacing in actions/cache/filebackend/filerepo/job folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: Idb93d34e314e5f314223b79208968d6bcd30c40e
2013-04-20 19:18:13 +02:00
umherirrender
9d19d7342a Remove a bunch of trailing spaces and unneeded newlines
Change-Id: I7db616db8c969567d420c0161fa207b366e292b6
2012-10-19 22:03:05 +02:00
Reedy
68f50015c7 Improved/added parameter documentation
Change-Id: Ie48852acb434f6b62fda0b13b13e076f833d8975
2012-05-18 03:58:15 +01:00
Alexandre Emsenhuber
d5381d0ea0 Added missing GPLv2 headers in some places.
Also made file/class documentation more consistent.

Change-Id: Iad15427368d13a6cc11c8512b7265e8c7b8878a5
2012-05-07 09:11:33 +02:00
Aaron Schulz
c8e1463cc0 [FileRepo] Various code cleanups.
* Made File::isHashed() wrap FileRepo::getHashLevels(). Removed now-used FileRepo::isHashed().
* Removed FileRepo::simpleClean(). Not useful anymore since the paths in Status errors don't have $IP or upload dirs anymore.
* Removed code in FileRepo::fileExistsBatch() and FileRepo::cleanupBatch() to handle FS file paths, which should never be passed in anymore. Likewise, removed FILES_ONLY parameter.
* Removed FileRepo::append()/appendFinish() stub functions.
* Added FileRepo::assertWritableRepo() function to better handle repos that are read-only by design rather than the hack of overwriting each function (several were missed).
* Added FileBackend::isPathTraversalFree() function and used it in FileRepo::validateFilename() to avoid duplication.
* Tweaked FileRepo::freeTemp() to avoid file locking and made FileRepo::cleanupBatch() return a Status.
* Moved FileRepo::cleanupDeletedBatch() near FileRepo::deleteBatch().
* Added type hinting to a few places.
* Tweaked some misleading doc comments and added function visibility markers.

Change 1:
* Simplified NullRepo to also use assertWritableRepo(). It is currently only used by a single unit test.

Change-Id: I1cd0f4971011772e38e5156f94ffc50325372f28
2012-04-05 12:48:47 -07:00
Sam Reed
76246b9bf5 More return documentation 2012-02-09 21:33:27 +00:00
Tim Starling
c1f96d1530 Revert the dbname -> dbName part of r90430. dbname actually dates back to r32578 and is referenced in the documentation for $wgDBservers, so it can't be easily changed. We can still kill tablename though, it's not too late. 2011-06-20 07:40:07 +00:00
Tim Starling
8371ede86b Fixes for r80864 for 1.18 backport:
* Use camel case in tableName and dbName, dontrunallyourwordstogetherlikethis
* Call the function factory() instead of newFromType(), for consistency with the rest of MediaWiki
2011-06-20 07:00:50 +00:00
Merlijn S. van Deen
3947a80b1c Fixes ForeignDBRepo SQL server connection. Fixes bug #28836. 2011-05-05 15:04:19 +00:00
Chad Horohoe
b3e99eec5b Followup to r79848 (and really, make it useful...)
Turn DatabaseBase::classFromType() into newFromType() factory function for constructing a new object based on a given type and (optional) params. Documented it fairly clearly.

I think it looks nicer :)
2011-01-24 16:31:36 +00:00
Chad Horohoe
97b62e3034 Add DatabaseBase::classFromType() to reduce the 'Database' . $type duplication 2011-01-07 22:32:52 +00:00
Sam Reed
783d4ab681 Simplest possible fix for *(bug 26552) ForeignDBRepo broken?
Going to see what other uses I can find of stuff constructing their own Database Instances...
2011-01-04 19:33:13 +00:00
Alexandre Emsenhuber
4f1764f7a0 * added file description headers
* changed a @ingroup Media to FileRepo
2010-09-04 18:13:18 +00:00
Conrad Irwin
b005164c9b Revert r64834, hasty and unnecessary 2010-04-09 22:09:07 +00:00
Conrad Irwin
720730cec2 Bug 23120. Make sure that all wikis sharing the same database use initialCapital 2010-04-09 21:31:16 +00:00
Alexandre Emsenhuber
5b7d86d07f Fixed some doxygen warnings 2010-03-26 21:55:13 +00:00
Tim Starling
6cba5762b7 Bug 19240 (bad image list performance regression):
* Don't connect to the commons DB on cache hits, in order to determine the cache key name. Removed remnants of that bright idea from GlobalFunctions.php.
* Fixed total failure of negative caching in checkRedirect() due to memcached stripping trailing spaces from string values. Probably never worked.

Also:
* Respect hasSharedCache in foreign repos. Recently-added code was apparently ignorant of this setting.
* Renamed getMemcKey() to getSharedCacheKey() to make its function more clear, introduced getLocalCacheKey() to do the other thing. Fixed its parameters to be like wfMemcKey() and used it in more places.
* Used getLocalCacheKey() in various places instead of wfMemc(), to avoid having multiple repositories overwrite each others' caches. 
* Fixed the BagOStuff bug that the FIXME in LocalRepo::checkRedirect() appears to refer to.
* Removed getMasterDB() and getSlaveDB() from FileRepo, it's incorrect to assume that a repo other than a LocalRepo has an associated database.
* Made FileRepo::invalidateImageRedirect() a stub, to match FileRepo::checkRedirect(). Moved the functionality to LocalRepo where checkRedirect() is concretely implemented.
2009-06-17 07:31:00 +00:00
Bryan Tong Minh
8adc41248d Forgot to remove the newFileFrom row thingies from ForeignDBRepo 2008-05-30 18:19:02 +00:00
Bryan Tong Minh
57af2aecb5 * Use a proper factory for newFromRow
* Don't remove found files from the $titles parameter in findFiles
2008-05-30 13:16:08 +00:00
Brion Vibber
758e0bc7f9 *cough* thx 2008-05-20 18:49:11 +00:00