Commit graph

276 commits

Author SHA1 Message Date
Brad Jorsch
d3642413a1 SECURITY: Fix handling of CommentStore and insertSelect
CommentStore->insert() takes the raw comment, not quoted, and returns
fields appropriate for passing as $a to IDatabase->insert() or $values
to ->update(). Such fields need to be passed through
IDatabase->addQuotes() to be appropriate for passing in $varMap to
IDatabase->insertSelect().

Change-Id: Ie64b279ee7cf9c8c396af385e46c826e0597ab1e
2017-09-19 09:32:29 +10:00
Reedy
ab4e6fdd51 Return description fields for unprefixed image cache rows
Bug: T175444
Change-Id: I5560187d3850253095b695dc7a3cfc954fba9318
2017-09-18 14:13:05 +00:00
Brad Jorsch
33ba36c88a Replace more problematic uses of "SELECT *"
With the introduction of CommentStore, selects from various table
require certain joins or column aliases for proper operation.

Change-Id: I2ecb6030bf253664c3227d69b5ed2277596a83f0
2017-09-13 13:09:51 +00:00
Brad Jorsch
035f4635ef Use CommentStore::getCommentLegacy with CommentStore::getFields
It doesn't matter when $wgCommentTableSchemaMigrationStage is
MIGRATION_OLD, but it'll fail when we start changing that to later
migration stages.

Follows up I3447a412.

Change-Id: I6c010cefedd99324080ec078b83159d12709c1b1
2017-09-11 10:33:29 -04:00
Kunal Mehta
8e87dd16b8 LocalFile: Fix setting $this->description after CommentStore changes
Bug: T175444
Bug: T175443
Change-Id: I3447a4129300c2c7dc0d585404e7747b6a5e2b77
2017-09-09 19:40:23 +00:00
jenkins-bot
a17d459d4b Merge "Handle comment truncation in CommentStore" 2017-09-05 18:39:21 +00:00
Kunal Mehta
0cb2a09b6d Always put type information before variable name for @var tags
Like other tags, @var should have the type information before the
variable name.

<https://docs.phpdoc.org/references/phpdoc/tags/var.html>

Change-Id: I9eca6957b1990fa8cc687103dc02ee38af5f9086
2017-09-04 11:06:51 -07:00
Brad Jorsch
6ec1a31502 Handle comment truncation in CommentStore
Since the caller doesn't (and shouldn't) know whether CommentStore is
using the old or the new schema, it should leave truncation of comments
to CommentStore.

Change-Id: I92954c922514271d774518d6a6c28a01f33c88c2
2017-09-01 15:03:45 -04:00
Brad Jorsch
11cf01dd9a Add comment table and code to start using it
A subsequent patch will remove the old columns.

Bug: T166732
Change-Id: Ic3a434c061ed6e443ea072bc62dda09acbeeed7f
2017-08-30 15:05:00 +10:00
Thiemo Mättig
d14faa6bed Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

Change-Id: I994d11e05f202b880390723e148d79c72cca29f0
2017-07-10 10:15:51 +00:00
Gilles Dubuc
b8292cef01 Make file purging also purge old versions
Also fixes purging for repos with sha1 thumb URLs.

Bug: T169198
Change-Id: Ibb98ecce83d690cc46769644038b54e37aea0b0d
2017-06-30 10:49:25 +02:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
jenkins-bot
aae0c8d425 Merge "Perform input validation for file $page at some strategic locations" 2017-06-26 06:55:16 +00:00
Gilles Dubuc
c2ea52a545 Remove X-Content-Dimensions header
Reverts 84e4d75088, and parts of cdfe08439c and 4511f6fa9f.

Bug: T150741
Bug: T167034
Change-Id: I39cfcf2cb231b6dfef569968fba6f473da258916
2017-06-12 18:07:28 +00:00
Gilles Dubuc
2cab7c0f91 Process content headers when metadata is empty
The new width/height override means that files without
any metadata can get content headers.

Bug: T150741
Change-Id: I92199a4aa9e0408572ebf7d8e4c04cd0196f0d0a
2017-06-02 17:12:09 +02:00
Gilles Dubuc
84e4d75088 Use file width/height instead of metadata for getContentHeaders
This allows us to populate X-Content-Dimensions without touching the
existing metadata format. Which makes the migration of existing content a lot faster by
only having to run refreshFileHeaders.

Bug: T150741
Change-Id: I2c0f39b2b01f364c3fab997ccc2f874b7f101d8a
2017-05-23 19:18:58 +00:00
Gilles Dubuc
cdfe08439c Store original media dimensions as additional header
For storage repos that support headers (such as Swift), this will store the original
media dimensions as an extra custom header, X-Content-Dimensions.
The header is formatted to minimize its length when dealing with multipage
documents, by expressing the information as page ranges keyed by dimensions.

Example for a multipage documents with some pages of different sizes:
X-Content-Dimensions: 1903x899:1-9,11/1903x873:10

Example for a single page document:
X-Content-Dimensions: 800x600:1

Bug: T150741
Change-Id: Ic4c6a86557b3705cf75d074753e9ce2ee070a6df
2017-05-09 15:49:28 -07:00
Arlo Breault
e430aa19ab Perform input validation for file $page at some strategic locations
* Not a holistic solution, but these are the places that currently
   assume a natural number for $page.

Bug: T100453
Change-Id: Ic611e5379dc8d138034b62ff8d678dfc68ced023
2017-05-03 21:40:39 -04:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Timo Tijhof
0c5b6ecc8f Clean up get_class() in /includes/filerepo and /includes/resourceloader
* get_class()        -> __CLASS__ (same as self::class)
* get_called_class() -> static::class
* get_class($this)   -> static::class

Change-Id: Ic4f9b150ddc35e25f51297de7fc12410d421be4d
2017-03-07 21:30:29 +00:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Brad Jorsch
fb3ae6fbe3 Replace use of &$this
Use of &$this doesn't work in PHP 7.1. For callbacks to methods like
array_map() it's completely unnecessary, while for hooks we still need
to pass a reference and so we need to copy $this into a local variable.

Bug: T153505
Change-Id: I8bbb26e248cd6f213fd0e7460d6d6935a3f9e468
2017-01-31 23:01:54 -05:00
Aaron Schulz
2e5eb693de objectcache: add WANObjectCacheReaper for assuring purges
* This fixes keys based on some sort of change log.
  Updates are wrapped in a mutex and keep track of the
  last known good position.
* Make WANObjectReapUpdate class that cleans up title
  related keys using the recentchanges table. This triggers
  as a deferred updates on RC view.

Change-Id: I7f14b9ca2533032147e62b1a3cc004a23da86579
2017-01-27 02:21:06 +00:00
Aaron Schulz
c4d275b7d5 Remove FileRepoStatus references
Change-Id: I03190273670f5c255423cf59019cbf12220c5498
2016-12-21 19:07:33 +00:00
Aaron Schulz
f525c72590 Rename getSlaveDB() FileRepo method to getReplicaDB()
The old name is left as an alias.

Change-Id: I60ab2cd5ce05df4247d5e25b017d2debee56554e
2016-11-18 07:42:39 -08:00
umherirrender
25d87429e6 Refactor rev_delete handling on deletion
Refactoring handling of the rev_delete database field, which is not
needed to select, when the suppress option is set.

Also added a new const, to avoid a comment about that things should be
15 ...

Change-Id: Ia5bcb835172bf6c10e2e872555fc67e91806ebe1
2016-10-27 19:33:14 +02:00
Aaron Schulz
8585cc9ffb Fix numerous FileRepo IDEA warnings
Change-Id: I3522f37b675efffb68dec9d125faacd8c8776e64
2016-10-05 16:38:15 -07:00
Aaron Schulz
5bbac35ed1 Replace DatabaseBase:: with Database:: and update type hints
Change-Id: I3919b04eb2de4fa0bf8a02239fb5bbf17d347511
2016-09-27 04:20:03 +00:00
Aaron Schulz
ff5abb66b4 Migrate callers to new MWFileProps::getPropsFromPath() method
* FSFile should not be responsible for handling this much logic.
* Make more MediaHandler classes aware of the fact that an object
  other than File might be passed in. Use the FSFile instead of a
  useless empty stdClass object.
* Also added more fields to FSFile::placeholderProps to make it
  more complete.

Change-Id: I9fe764b2a7261af507c6555e6a57273cf7d00d36
2016-09-22 00:33:46 +00:00
Aaron Schulz
5daf81eb20 Fixed method name casing and IDEA warnings in FileRepo
Change-Id: I463885a9f425491cd040cb1cdbc4c16925522848
2016-09-19 07:54:52 +00:00
Aaron Schulz
37ad84cb91 Cleanups and IDEA warning fixes to FileRepo classes
Change-Id: I98247b299c31b93df183981adc517187cb4aefe5
2016-09-19 06:34:24 +00:00
Aaron Schulz
a2790b1b80 Remove wfGetCaller() dependency from DatabaseBase
Change-Id: I3e240b2eb5c1f6a21f1bc974c3d28f5755c7451a
2016-09-15 14:40:00 -07:00
umherirrender
1bc2b88bb2 Use Database::timestamp in LocalFile::loadFieldsWithTimestamp
Postgres needs a special timestamp format on sql which is handled by
Database::timestamp.

Bug: T145066
Change-Id: I8db958ecbea7c12af8caf8354490e59500975e4b
2016-09-10 22:36:50 +02:00
Aaron Schulz
6ec1bcc04a Convert LocalFile to using getWithSetCallback() for caching
Changed the hashing for the keys to SHA-1, which also avoids
problems with old MediaWiki versions seeing the new WAN
versioned keys.

Also fixed a few annoying IDEA errors

Change-Id: Ie608fb86421bc96e05e4a3b352f39b4938a243e4
2016-08-31 17:36:02 -07:00
Aaron Schulz
52511952de objectcache: add and use adaptiveTTL() method
* This better handles delayed/lost cache purges by
  having lower TTLs for entries that often changes.
* Use this for foreign upload description page caches,
  we purges are never received from the source wiki.
* Also use this for User and LocalFile cache TTLs.
* Also move the Database::getCacheSetOptions() call in
  User *before* doing the queries, which is preferred.
* Fixed some IDEA errors too, like the undeclared
  mApiBase field.

Change-Id: I70f8ebb29ac853c2a530d9eedb9e7facc1b7b710
2016-08-31 03:40:56 +00:00
Aaron Schulz
e2c03a8afd Avoid INSERT..SELECT in LocalFileDeleteBatch
That construct has poor locking characteristics in terms of
auto-inc columns as well as not allowing such inserts concurrently
for statement-based replication. Also, the INSERT..SELECT did not
have an ORDER BY, which could lead to fa_id drift with statement
based replication.

Change-Id: Iaacb75d9931b4cd24b70bdcaadd0e3979c7e9c90
2016-08-23 10:10:52 -07:00
Brian Wolff
5c24203c6e SECURITY: Do not allow undeleting a revdel'd file if its top file
This prevents admins being able to view suppressed files, by simply
deleting them, and then undeleting only the file revision that they
want to view.

This dates back to r43288. Unclear if it was intentional.

Bug: T132926
Change-Id: Ib767de853a37099305db20529378fa756ee1bdfe
2016-08-23 02:03:56 +00:00
Gilles Dubuc
7e0d148f57 Move thumbnail rendering to a more appropriate spot
At the moment the job might start before the transaction
that creates the file's row in the DB has had a chance
to run.

Bug: T106740
Change-Id: If5b94e83d8bbcc6aebfe7193f7b580f03cbd627d
2016-08-16 14:19:20 +02:00
Aaron Schulz
c95583a560 Defer maybeUpgradeRow() post-send since they can trigger on non-POST
Change-Id: I791e7133d49ed9cd6dd40bb3fa35ea38d1ceba10
2016-08-11 09:19:09 -07:00
Aaron Schulz
29164bbf5a Make recordUpload2() deferred updates PRESEND again
Bug: T141687
Change-Id: Ica1a09a6e580b9da659d22183f396d88f694f256
2016-07-31 19:55:45 -07:00
jenkins-bot
d4f1e00a0c Merge "Make revision deletion acquire file locks to avoid races" 2016-07-27 00:47:03 +00:00
Aaron Schulz
d217cdf4b5 Make revision deletion acquire file locks to avoid races
Also made RevisionListBase an Iterator to avoid ugly loops here

Change-Id: I40d2d8cf63df95c59d0e1275e3ec45aff238e1cb
2016-07-26 16:32:15 -07:00
Aaron Schulz
3a922d36d8 Convert onTransactionIdle() callers to DeferredUpdate subclasses
* This puts the complex logic here after the commit step for
  all DBs, making the main multi-DB transaction more likely
  to be atomic. Previously, the idle callbacks could be hit
  bewteen DB commits.
* Enforce transactionality via AtomicSectionUpdate.
* Use $this instead of $that hacks for old PHP versions.

Change-Id: Idf7d54fdac6487f86907099680f5c1c4f5530b4e
2016-07-25 16:49:15 -07:00
Aaron Schulz
ad95e01823 Avoid use of DB rollback() in LocalFileMoveBatch
* Verify the DB updates and bail before doing anything instead
  of relying on rollback() if something does not match up.
* Do the file copying before updating the DB so that there is
  nothing to rollback if they fail.
* Improved failCount in case the current version is missing.
  It should also reflect all the missing old versions.

Change-Id: Ie2d316548d8e5584cc69bb9f1425db108b05be5c
2016-07-25 22:16:20 +00:00
Aaron Schulz
6f415f73bb Avoid the use of DB rollback() in LocalFileDeleteBatch
Race conditions are already handled by LockManager now.

Change-Id: Idb9ac511d565db4920aff7faa6ff291e05079798
2016-07-22 05:03:54 +00:00
Aaron Schulz
3c21f0e8e5 Add AutoCommitUpdate class and replace some onTransactionIdle callers
* This puts the complex logic here after the commit step for
  all DBs, making the main multi-DB transaction more likely
  to be atomic.
* Made some cleanups to AtomicSectionUpdate and made it cancel
  if the transaction is rolled back as it should.
* Also cleaned up some closures for PHP 5.4.

Change-Id: If2f7bb6b1ba6daf1cfdc934f27c32b0b10431a3d
2016-07-21 05:24:28 +00:00
Aaron Schulz
a66544f2cf Make LocalFile::lock() initialize DBO_TRX transactions
If the first query to the master DB is after lock() and DBO_TRX is
set, make sure that the LocalFile updates still join the implicit
transaction that the rest of the request is in.

This helps keep the commit step tight when multiple DBs are touched
by making sure that the main DB commits in commitMasterChanges()
along with any others.

Bug: T119736
Change-Id: I6cc29f9201947e4415336528d30cba7f88567b41
2016-07-14 23:30:25 +00:00
Aaron Schulz
d70b081011 Fixes to LocalFile::lock()
* Added onTransactionResolution() DB method.
* Use this method so that file unlocks fire on unlockAndRollback()
  as well as on DB errors (via MWExceptionHandler::handleException).
  This prevents locks from getting stuck for minutes when deadlocks
  happen, since the LockManager::destruct() method is not reliable.
* Fix broken reference counting which always released locks on the
  first unlock() call, even if there were 2+ lock() calls.
* Added some type hints to IDatabase methods.
* Fixed DatabaseBase::__destruct() logging to include all callbacks.

Bug: T132921
Change-Id: I684706957f4d794cb6fe61505b0d26b7893de706
2016-07-04 18:11:55 -07:00
Aaron Schulz
e43fe2a3e8 Use a log group for LocalFile lock errors
Bug: T132921
Change-Id: I55cafc9e2fff302e55d40a678bf7e25b4bf24026
2016-06-29 17:22:03 -07:00