Commit graph

239 commits

Author SHA1 Message Date
Brian Wolff
5c24203c6e SECURITY: Do not allow undeleting a revdel'd file if its top file
This prevents admins being able to view suppressed files, by simply
deleting them, and then undeleting only the file revision that they
want to view.

This dates back to r43288. Unclear if it was intentional.

Bug: T132926
Change-Id: Ib767de853a37099305db20529378fa756ee1bdfe
2016-08-23 02:03:56 +00:00
Gilles Dubuc
7e0d148f57 Move thumbnail rendering to a more appropriate spot
At the moment the job might start before the transaction
that creates the file's row in the DB has had a chance
to run.

Bug: T106740
Change-Id: If5b94e83d8bbcc6aebfe7193f7b580f03cbd627d
2016-08-16 14:19:20 +02:00
Aaron Schulz
c95583a560 Defer maybeUpgradeRow() post-send since they can trigger on non-POST
Change-Id: I791e7133d49ed9cd6dd40bb3fa35ea38d1ceba10
2016-08-11 09:19:09 -07:00
Aaron Schulz
29164bbf5a Make recordUpload2() deferred updates PRESEND again
Bug: T141687
Change-Id: Ica1a09a6e580b9da659d22183f396d88f694f256
2016-07-31 19:55:45 -07:00
jenkins-bot
d4f1e00a0c Merge "Make revision deletion acquire file locks to avoid races" 2016-07-27 00:47:03 +00:00
Aaron Schulz
d217cdf4b5 Make revision deletion acquire file locks to avoid races
Also made RevisionListBase an Iterator to avoid ugly loops here

Change-Id: I40d2d8cf63df95c59d0e1275e3ec45aff238e1cb
2016-07-26 16:32:15 -07:00
Aaron Schulz
3a922d36d8 Convert onTransactionIdle() callers to DeferredUpdate subclasses
* This puts the complex logic here after the commit step for
  all DBs, making the main multi-DB transaction more likely
  to be atomic. Previously, the idle callbacks could be hit
  bewteen DB commits.
* Enforce transactionality via AtomicSectionUpdate.
* Use $this instead of $that hacks for old PHP versions.

Change-Id: Idf7d54fdac6487f86907099680f5c1c4f5530b4e
2016-07-25 16:49:15 -07:00
Aaron Schulz
ad95e01823 Avoid use of DB rollback() in LocalFileMoveBatch
* Verify the DB updates and bail before doing anything instead
  of relying on rollback() if something does not match up.
* Do the file copying before updating the DB so that there is
  nothing to rollback if they fail.
* Improved failCount in case the current version is missing.
  It should also reflect all the missing old versions.

Change-Id: Ie2d316548d8e5584cc69bb9f1425db108b05be5c
2016-07-25 22:16:20 +00:00
Aaron Schulz
6f415f73bb Avoid the use of DB rollback() in LocalFileDeleteBatch
Race conditions are already handled by LockManager now.

Change-Id: Idb9ac511d565db4920aff7faa6ff291e05079798
2016-07-22 05:03:54 +00:00
Aaron Schulz
3c21f0e8e5 Add AutoCommitUpdate class and replace some onTransactionIdle callers
* This puts the complex logic here after the commit step for
  all DBs, making the main multi-DB transaction more likely
  to be atomic.
* Made some cleanups to AtomicSectionUpdate and made it cancel
  if the transaction is rolled back as it should.
* Also cleaned up some closures for PHP 5.4.

Change-Id: If2f7bb6b1ba6daf1cfdc934f27c32b0b10431a3d
2016-07-21 05:24:28 +00:00
Aaron Schulz
a66544f2cf Make LocalFile::lock() initialize DBO_TRX transactions
If the first query to the master DB is after lock() and DBO_TRX is
set, make sure that the LocalFile updates still join the implicit
transaction that the rest of the request is in.

This helps keep the commit step tight when multiple DBs are touched
by making sure that the main DB commits in commitMasterChanges()
along with any others.

Bug: T119736
Change-Id: I6cc29f9201947e4415336528d30cba7f88567b41
2016-07-14 23:30:25 +00:00
Aaron Schulz
d70b081011 Fixes to LocalFile::lock()
* Added onTransactionResolution() DB method.
* Use this method so that file unlocks fire on unlockAndRollback()
  as well as on DB errors (via MWExceptionHandler::handleException).
  This prevents locks from getting stuck for minutes when deadlocks
  happen, since the LockManager::destruct() method is not reliable.
* Fix broken reference counting which always released locks on the
  first unlock() call, even if there were 2+ lock() calls.
* Added some type hints to IDatabase methods.
* Fixed DatabaseBase::__destruct() logging to include all callbacks.

Bug: T132921
Change-Id: I684706957f4d794cb6fe61505b0d26b7893de706
2016-07-04 18:11:55 -07:00
Aaron Schulz
e43fe2a3e8 Use a log group for LocalFile lock errors
Bug: T132921
Change-Id: I55cafc9e2fff302e55d40a678bf7e25b4bf24026
2016-06-29 17:22:03 -07:00
Aaron Schulz
471ab05ea2 Make LocalFileLockError an error page exception
This presents a better message than a cryptic red exception hash box

Bug: T132921
Change-Id: Ie3f358378af54d0348f18cfb1df763a182259906
2016-06-11 03:00:09 +00:00
Aaron Schulz
a019722c98 filerepo: Add missing quote back to LocalFileLockError message
Follows-up e724d14495.

Change-Id: I48e304a307d3c0427f797a382dc23347bb7e914f
2016-05-25 18:27:22 +00:00
Aaron Schulz
e724d14495 Add the LockManager error to LocalFileLockError exceptions
Change-Id: Ibfa7312993806554f7939b4f5692e30714f71304
2016-05-24 00:06:36 -07:00
Aaron Schulz
e702a70082 Increase LocalFile lock() timeout and improve error message
Bug: T132921
Change-Id: Iaec861475b4b6ec91a824672fb81ed5a8eb2b6a9
2016-05-12 15:52:56 -07:00
Aaron Schulz
dad254dfab Avoid lock error exceptions during upgradeRow() contention
Bug: T132921
Change-Id: I229031c3d4ae5b700fcc4d4dd3f5208a853823dc
2016-04-29 21:34:18 +00:00
jenkins-bot
8cb8af442a Merge "Use english messages for background use of Status::getWikiText" 2016-04-13 14:03:56 +00:00
Ori Livneh
d721dc49c8 Don't report image cache hits / misses
This single metric is responsible for a substantial proportion of overall
statsd traffic on the Wikimedia cluster, per the task cited below. Nobody is
closely monitoring this statistic. We can reintroduce it with sampling at a
later date if it is deemed to be useful.

Bug: T101141
Change-Id: I6457cb91d527ce42c0918ac4e80613e6b59061ac
2016-04-12 11:30:21 -07:00
umherirrender
932c37e3cb Use english messages for background use of Status::getWikiText
Status::getWikiText is used for internal logging, api error messages and
maintenance scripts. All this places are usually in english, so pass an
english language to getWikiText.

Change-Id: I3010fca8eb5740a3a851c55a8b12e171714c78f7
2016-04-12 20:01:44 +02:00
jenkins-bot
6a4214dc89 Merge "Allow to pass a rev id to a log entry without making it unpatrolled" 2016-04-07 08:47:03 +00:00
Aaron Schulz
7889d3f150 filerepo: Mark some internal LocalFile methods private
Change-Id: I1c9ab1e1460b5c66e8a1dbdb387f80004eaba994
2016-03-31 00:45:44 +00:00
Reedy
b5656b6953 Many more function case mismatches
Change-Id: I5d3a5eb8adea1ecbf136415bb9fd7a162633ccca
2016-03-19 00:20:58 +00:00
cenarium
d7a37e8daa Allow to pass a rev id to a log entry without making it unpatrolled
This adds new methods to ManualLogEntry that allow to specify that
a log entry is patrollable. RecentChange::newLogEntry then checks
this in order to allow or not unpatrolled status instead of checking
if there is an associated rev id. This allows to associate a revision
to a log entry without possibly making the recent change unpatrolled,
and extensions can implement patrol of actions not dependent on an
associated revision id.

Bug: T127848
Change-Id: I98298047142819b69639e6ca9d77c5ba982a380c
2016-03-14 16:11:05 +01:00
Aaron Schulz
d8d4b26c59 Move some LocalFile comments around
Change-Id: I7d7c5618481245aa6e8328ad58f5e841f010fe01
2016-03-11 14:14:40 -08:00
jenkins-bot
a25d6fbc28 Merge "Make UploadBase use TempFSFile to wrap the temporary file" 2016-03-10 20:04:54 +00:00
Aaron Schulz
dcb5ec5cbf Make UploadBase use TempFSFile to wrap the temporary file
* The cleanupTempFile() method now only marks it as ready
  for unlinking when unreferenced.
* Pass TempFSFile down the FileRepo call chain so that it
  can build off a5d903860a and allow more fast async writes
  for secondary backends. Previously, the 'store' operation
  used on upload forced sync file change replication writes.
* Also fix bogus method call in LocalFile::publishTo().

Bug: T91869
Change-Id: I06caa6e5d8bdec9a7cae2b68cb02dd2e64b9ac74
2016-03-07 20:54:06 +00:00
Geoffrey Mon
e70c4eb664 Add tags support to patrol, protect, unblock, and undelete
- Add 'tags' parameters to appropriate API modules
- Add tag-adding logic to appropriate functions that carry out
  relevant functions
- ManualLogEntry::{set,get}Tags to handle adding tags to log
  entries in a cleaner fashion
- Use ManualLogEntry::setTags in LocalFile::recordUpload2

Bug: T97720
Change-Id: I98c52da7985623bfdafda2dc2dae937b39b72419
2016-02-29 16:59:31 -05:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Bartosz Dziewoński
b40bfd30c5 Document how LocalFile::upload() and UploadBase::performUpload() ignore user permissions
Follow-up to 61c7852049.

Change-Id: I6e69b1fa12f336dc4dd12e845cb8bf42abd599b2
2016-01-26 18:03:56 +01:00
jenkins-bot
6389f3fb53 Merge "Make it possible to tag new file uploads without messy queries" 2016-01-25 21:00:13 +00:00
Bartosz Dziewoński
61c7852049 Make it possible to tag new file uploads without messy queries
UploadBase::performUpload() now takes a $tags parameter and passes it
to LocalFile::upload() and LocalFile::recordUpload2(), which
eventually adds the requested tags to the log_id, rev_id and rc_id
that are created for the file upload.

Previously you'd have to query the database for the latest rev_id and
log_id for the page title under which the title is being uploaded, as
performUpload() is unable to return them to you because it's all
deferred in funny ways.

Bug: T121874
Change-Id: I99a8fd67c84219d2715d3d88cc21500614431179
2016-01-25 20:06:53 +01:00
Bartosz Dziewoński
e09772cc56 LocalFile: Ensure same timestamp for log entry and image revision
For file patrolling, we rely on the timestamp of image revision
(`image` table) and log entry (`logging` table) being the same, but we
weren't actually making sure it is the same.

Note that the timestamp of page revision can still differ, but that
shouldn't matter and it's a more involved fix.

Change-Id: I8e1b75d744c4a96cb0bb636b153d6e8ea2ee1fcf
2016-01-24 21:31:34 +01:00
victorbarbu
df622c4195 Provide short URL to file description page in imageinfo API
Bug: T122439
Change-Id: I0f00b986e6095bdb9b8d6af6fbc5b01995227e02
2016-01-12 15:48:52 +02:00
Bartosz Dziewoński
f66ff60244 LogEntry: Make associated rev id persistent (store in logging and log_search)
The implementation from 0160b410bd only
stored the associated rev id in the `recentchanges` table, which was
sufficient for its purpose of allowing patrol of these log entries and
provided some change tagging improvements, but ephemeral.

Now we also store it in `logging` and `log_search` tables, which makes
it possible to find a revision for the log entry and vice versa.

* For `log_search` we just add a row with 'associated_rev_id' as key,
  it was designed for exactly this purpose.
* For `logging` we store it in log_params field in ManualLogEntry, and
  transparently remove it from there when building a DatabaseLogEntry.

Bug: T122089
Change-Id: I559587ff77b3c94b31a6c5951287b6f9c1e167ce
2016-01-07 02:19:51 +00:00
cenarium
c53f647658 Allow patrol of uploads
This allows to patrol file uploads, both new files and new file
versions, from the description page, provided $wgUseFilePatrol
is set to true. Special:NewFiles can be filtered to hide patrolled
files.

Bug: T11501
Change-Id: If71af58719a4461f12d125455b7bef07164525ca
2016-01-06 14:57:22 -08:00
Aaron Schulz
6af3c39c07 Replace "squid" with "CDN" in various comments
Change-Id: Idcc528daf28e119349155d36e30a9bcf61b2e7d5
2015-12-09 17:35:37 -08:00
Aaron Schulz
282c5fa9f3 Rename SquidUpdate => CdnCacheUpdate
Squid is not the only possible CDN

Change-Id: Ie2a2955847c5706e630322bbbab71c9d063b378f
2015-12-09 16:31:17 -08:00
Aaron Schulz
afbff42aca Make CDN purge calls use DeferredUpdates
* Using addUpdate() makes sure purges are coalesced and
  de-duplicated.
* Also removed incosistent $wgUseSquid checks. If CDN caching
  is not used, then $wgSquidServers will just be empty anyway.

Bug: T119016
Change-Id: I8b448366f037f668385d252f9d68289b71d1a707
2015-12-04 19:09:03 +00:00
Aaron Schulz
bbc8f40575 Move down $logEntry publish() call in recordUpload2()
This was moved before the defered doEditContent() call
for new pages by mistake.

Bug: T118433
Change-Id: I007acf82622ec6d68c007921f632793392025f40
2015-11-11 19:36:16 -08:00
Aaron Schulz
8c1d27c1d7 Remove spammy wfDebug() from LocalFile::loadFromCache()
Change-Id: I36ad31f197ec8f71c97e6a8673ac5ca979f8d9ab
2015-11-11 13:36:53 -08:00
Aaron Schulz
a62f165013 Always defer the log publish() in recordUpload2
This is more performant and simplifies the code a bit.

Change-Id: I7a30e000ae4eff47519b83eb6bdde57eeb8a18f6
2015-11-06 16:15:34 -08:00
Aaron Schulz
8e1f6d5506 Convert recordUpload2() to using startAtomic/endAtomic
* This avoids breaking transactions due to nesting
* Also improve readability a bit in this area

Change-Id: I81c41e83d14aa59930bfb99522ebcc25d8aa14f9
2015-11-05 18:33:38 +00:00
jenkins-bot
d9d5cbc8fc Merge "Remove begin/commit from recordUpload2()" 2015-10-07 02:31:39 +00:00
Aaron Schulz
d9725ffe16 Remove begin/commit from recordUpload2()
Use deferred updates and transaction hooks
to get the cache updates after the publish()
DB updates instead

Change-Id: I0a922182f7dd5f22c1e661171b0df1a49a637aef
2015-10-06 12:02:11 -07:00
Aaron Schulz
54758dd160 Change getCacheSetOptions() callers to use "Database"
* This is less verbose that DatabaseBase
* Also add a few WAN cache doc comments

Change-Id: I5b6de6d0ffa06753ea96c50b63db7dae796475dc
2015-10-05 22:06:46 -07:00
jenkins-bot
4f7d15021d Merge "Make WANObjectCache sets account for slave lag" 2015-10-06 00:11:22 +00:00
jenkins-bot
6571a887e0 Merge "Cleanups to MigrateFileRepoLayout" 2015-10-05 23:50:38 +00:00
Aaron Schulz
db0b9ef264 Make WANObjectCache sets account for slave lag
* This gets lag information that is useful when
  the calling code is about to run queries that
  will have their results cached.
* This is now used in place of trxTimestamp() for
  WANObjectCache set() and getWithSetCallback().
* The WAN cache will use a low TTL if the lag is
  too high to avoid caching stale data for weeks.
* Bumped MAX_COMMIT_DELAY as nothing enforces it.

Bug: T113204
Change-Id: I2a95b4088cff42d6e980351555f81a4b13519e24
2015-10-05 16:45:13 -07:00