Commit graph

86 commits

Author SHA1 Message Date
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
jenkins-bot
4b63ca7113 Merge "Set context on RedirectSpecialPage in MediaWiki.php" 2016-02-11 04:15:14 +00:00
umherirrender
4f2fe64283 Set context on RedirectSpecialPage in MediaWiki.php
Special page instance from SpecialPageFactory::getPage needs a context
set, to avoid using RequestContext::getMain in SpecialPage::getContext

Change-Id: I6e0828c6c9668c4ef10a729533190a5225b8782f
2016-02-04 20:14:03 +01:00
Brad Jorsch
2257fe4228 Revert "Remove SessionManager, temporarily"
This reverts commit 823db5d63dd5200d04c63da50ba6bf16f928e70b.

Change-Id: Ibb3e023e4eb6715295586dea87d0725c344a8271
2016-02-03 21:44:59 +00:00
Brad Jorsch
5083e810eb Remove SessionManager, temporarily
The plan here is to take it out of 1.27.0-wmf.12 and put it back in
1.27.0-wmf.13.

Since BotPasswords depends on SessionManager, that's getting temporarily
removed too.

This reverts the following commits:
* 6acd424e0d SessionManager: Notify AuthPlugin before calling hooks
* 4d1ad32d8a Close a loophole in CookieSessionProvider
* fcdd643a46 SessionManager: Don't save non-persisted sessions to backend storage
* 058aec4c76 MessageCache: Don't get a ParserOptions for $wgUser before the end of Setup.php
* b5c0c03bb7 SessionManager: Save user name to metadata even if the user doesn't exist locally
* 13f2f09a19 SECURITY: Fix User::setToken() call on User::newSystemUser
* 305bc75b27 SessionManager: Don't generate user tokens when checking the tokens
* 7c4bd85d21 RequestContext::exportSession() should only export persisted session IDs
* 296ccfd4a9 SessionManager: Save 'persisted' flag in session metadata
* 94ba53f677 Move CSRF token handling into MediaWiki\Session\Session
* 46a565d6b0 Avoid false "added in both Session and $_SESSION" when value is null
* c00d0b5d94 Log backtrace for "User::loadFromSession called before the end of Setup.php"
* 4eeff5b559 Use $wgSecureCookie to decide whether to actually mark secure cookies as 'secure'
* 7491b52f70 Call session_cache_limiter() before starting a session
* 2c34aeea72 SessionManager: Abstract forceHTTPS cookie setting
* 9aa53627a5 Ignore auth cookies with value 'deleted'
* 43f904b51a SessionManager: Kill getPersistedSessionId()
* 50c5256352 SessionManager: Add SessionBackend::setProviderMetadata()
* f640d40315 SessionManager: Notify AuthPlugin when auto-creating accounts
* 70b05d1ac1 Add checks of $wgEnableBotPasswords in more places
* bfed32eb78 Do not raise a PHP warning when session write fails
* 722a7331ad Only check LoggedOut timestamp on the user loaded from session
* 4f5057b84b SessionManager: Change behavior of getSessionById()
* 66e82e614e Fix typo in [[MediaWiki:Botpasswords-editexisting/en]]
* f9fd9516d9 Add "bot passwords"
* d7716f1df0 Add missing argument for wfDebugLog
* a73c5b7395 Add SessionManager

Change-Id: I2389a8133e25ab929e9f27f41fa9a05df8147a50
2016-02-01 22:06:49 +00:00
Aaron Schulz
cca0a7e135 Move MaxUserDBWriteDuration logic to LBFactory
Change-Id: If7231af24f10ae29b7137f147abb60a351c9127f
2016-01-28 17:31:27 -08:00
Aaron Schulz
c4c81ee5a4 Cleanups and IDEA error fixes to initializeArticle()
Avoid using magic Article methods (use WikiPage instead).

Change-Id: I4057f78f3cde2185a6301a3444367196641a8e26
2016-01-20 00:16:59 +00:00
Aaron Schulz
bab8764cb7 Add more preOutputCommit() comments
Change-Id: Ia0b4c37b391b8e5e77397d21458f9638b49ebf71
2016-01-17 22:21:26 +00:00
Aaron Schulz
543b063773 Distinguish read vs write mode Action classes
Bug: T123589
Change-Id: I3c8fab646d3bb8cd468e5b60c27f3c5d43f9f97c
2016-01-15 01:13:38 +00:00
jenkins-bot
052770c40b Merge "Use dedicated cookie for skipping CDN cache after initiating DB changes" 2016-01-13 03:16:40 +00:00
Brad Jorsch
a73c5b7395 Add SessionManager
SessionManager is a general-purpose session management framework, rather
than the cookie-based sessions that PHP wants to provide us.

While fallback is provided for using $_SESSION and other PHP session
management functions, they should be avoided in favor of using
SessionManager directly.

For proof-of-concept extensions, see OAuth change Ib40b221 and
CentralAuth change I27ccabdb.

Bug: T111296
Change-Id: Ic1ffea74f3ccc8f93c8a23b795ecab6f06abca72
2016-01-12 21:57:01 +00:00
Aaron Schulz
5e0162f919 Use dedicated cookie for skipping CDN cache after initiating DB changes
This sets a short lived cookie that more robustly takes on the role that the
session initiation in SubmitAction has. It can be used via Varnish VCL.

Bug: T121440
Change-Id: I6214a0bce4e8bbbecbc3f9513e7c852285b4a9db
2015-12-23 12:50:59 -08:00
Aaron Schulz
58cffd04b8 Pass __METHOD__ to load balancer commit/rollback methods
Change-Id: I3fd87908c2a493fae49de6a29efe79f8d433c112
2015-12-22 18:30:20 -08:00
Amir E. Aharoni
a94900fab0 Fix transaction-duration-limit-exceeded
* Fix the wording.
* Add PLURAL support.

Bug: T121821
Change-Id: I9cf27534a5b7a055a7c2544f6366dd0e40c5e9a3
2015-12-19 23:23:18 +02:00
Bartosz Dziewoński
7972a4e811 Add $query to JavaScript redirect info
Bug: T109724
Signed-off-by: Chad Horohoe <chadh@wikimedia.org>
Change-Id: I57a8f75067365d3da6388d2f8f7fe95ed5e6f310
2015-12-18 01:46:04 -08:00
csteipp
1461ff2ecf SECURITY: Make Special:MyPage and friends fake redirect to prevent info leak
This prevents a malicious person from using external resources on their
website to cause the victim's web browser to load
Special:MyPage -> User:Username, and then looking it up in the page hit
statistics in order to correlate IPs from the malicious person's server
log, with usernames on wiki.

This feature can be disabled with $wgHideIdentifiableRedirects.

Bug: T109724
Signed-off-by: Chad Horohoe <chadh@wikimedia.org>
Change-Id: Ia0e742dc92c77af4832174dfa24c6dcaa6ee80e9
2015-12-18 01:46:03 -08:00
Aaron Schulz
ec12adf649 Expand MediaWiki::main() $wgTrxProfilerLimits logging
Remove the filter for basic GET actions.

Change-Id: Iba2b233474321f3d097bf9dfcf90b0feec0312b6
2015-12-16 15:19:29 -08:00
Aaron Schulz
4302b0419d Rename getSquidURLs() => getCdnUrls()
Change-Id: I433acc7990a5fcefd0d2ff5b14ba33dec0424706
2015-12-11 16:40:35 -08:00
Aaron Schulz
6af3c39c07 Replace "squid" with "CDN" in various comments
Change-Id: Idcc528daf28e119349155d36e30a9bcf61b2e7d5
2015-12-09 17:35:37 -08:00
Aaron Schulz
f606fd8d5c Rename setSquidMaxage() => setCdnMaxage()
Also updated the OutputPage member variable.

Change-Id: I0b41c66da58c6126c1bd96170c02d084a067383c
2015-12-09 17:11:23 -08:00
Aaron Schulz
c30b049580 Add $wgMaxUserDBWriteDuration to limit user-generated transactions
This is a low level catch-all net for huge updates that still slip
through. Features that let users add/remove arbitrarily many rows
to lists of arbitrary size can easily result in high lag due to
strange usage patterns or deliberate attacks.

Also removed duplicate 'autochange-username' JSON key.

Bug: T95501
Change-Id: I58a91ca23cae528ef1954d2d78c8f0a90681983e
2015-12-04 13:08:52 -08:00
Aaron Schulz
34dd248e28 Add pre-send update support to DeferredUpdates
* PRESEND/POSTSEND constants can now be used in addUpdate()
  and addCallableUpdate() to control when the update runs.
  This is useful for updates that may report errors the client
  should see or to just get a head start on queued or pubsub
  based updates like CDN purges. The OutputPage::output() method
  can easily take a few 100ms.
* Removed some argument b/c code from doUpdates().
* Also moved DeferrableUpdate to a separate file.

Change-Id: I9831fe890f9f68f9ad8c4f4bba6921a8f29ba666
2015-12-04 19:08:27 +00:00
jenkins-bot
18ec10a358 Merge "API DB commit and sticky DC cookie fixes" 2015-11-23 00:43:59 +00:00
Aaron Schulz
85edec239e API DB commit and sticky DC cookie fixes
* Make sure the API commits DBs *before* sending the
  response, in case something goes south which would
  make a sent response wrong.
* Make sticky DC cookies trigger with api.php too.
* Make sure UseDC cookies do not end up with a prefix.
  VCL rules should not have to deal with having a DB
  name as a prefix or such. This was an oversight.

Change-Id: I9e4090ab15c1c1493b0589a710184745dac9b0c1
2015-11-19 12:33:28 -08:00
Aaron Schulz
85c0f85e92 Decouple ChronologyProtector from user sessions
* It now works for users without using sessions.
  Sessions should not be cluttered with things
  unrelated to authentication and tokens.
* Public services doing api.php requests on behalf
  of a users only need to set XFF headers (as normal)
  for position wait logic to trigger. They can opt out
  of ChronologyProtector via a new HTTP header
  "ChronologyProtection: false".
* Requests across subdomains, such as the SUL2 handshake
  for CentralAuth on account creation, now have position
  wait logic applied. This helps avoid anomolies were a
  row just written in the last request may not be seen.
* Use merge() to avoid rolling back master positions if
  the user has multiple tabs open and doing writes at once.
* $_SESSION global state is gone from ChronologyProtector.
* Cleaned up post-send LBFactory::shutdown() logic for
  avoiding master position writes with an explicit flag.
* Use 'replication' debug log group in more places.

Bug: T111264
Change-Id: Ib25d05994d62b25c2f89e67b7f51009c54f4bca8
2015-11-12 23:11:18 +00:00
Ori Livneh
8340400f27 Add Timing interface
* Add `Timing`, an interface which mimics the W3C User Timing API.
  It provides a canonical way to store and retrieve markers (timestamps)
  and measures (timestamps + duration).
* As the initial use-case, use it to record 'requestShutdown'.

Change-Id: I36b29162ffcc091406df025463b0e2797e52f19a
2015-11-03 15:05:37 -08:00
Aaron Schulz
a8590172c8 Make DeferredUpdates::doUpdates always commit per task
* All callers are either using commit already or would be fine
  using it (e.g. Maintenance scripts and JobRunner that have
  no real transaction open).

Change-Id: I9f54b27619da6dac2cb63d255995aabc4ee78002
2015-10-22 01:27:08 +00:00
Aaron Schulz
0477d11be6 Send X-Database-Lagged header to help with debugging
Change-Id: Ib654c49a61a1279091c5da92a2bef2ad1f992d23
2015-10-21 18:58:02 +00:00
umherirrender
977c810302 Remove empty line comments
Remove empty line comments as found by the
MediaWiki.WhiteSpace.SpaceBeforeSingleLineComment.EmptyComment sniff

Change-Id: I5d694f7a7d3bc97e16300ba03c60ad17f3c912a5
2015-10-14 09:46:44 +02:00
Aaron Schulz
1846e2dc15 Made LinksUpdate on edit use the job queue
* LinksUpdate is now an EnqueueableDataUpdate
  and can yeild a prioritzed refreshLinks job.
* DeferredUpdates::runUpdates() now takes an enqueue
  flag to try to use jobs. This is set in restInPeace().
  Updates that change many links will be less likely to
  increase lag, as the runners are more strict about that.
* Also made the LinksDeletionUpdate job enqueue happen
  post-send on page deletion for consistency

Bug: T95501
Change-Id: I8863caef9c8f03234699d33e4d47d2310a0c8446
2015-10-08 01:02:03 -07:00
Aaron Schulz
c7b932af6b Lower CDN cache TTL when slave lag is high
* $wgCdnMaxageLagged controls exactly what that TTL is
  and the usual "max lag" settings determine what "high"
  is for lag (which already makes the site read-only).
* This helps avoids stale content getting stuck in CDN
  for a month just because a slave was lagged for a minute.
  Of course race conditions with normal slave lag and WAN
  cache relay purges can still lead to this problem, though
  the scope of it is reduced.

Bug: T113204
Change-Id: I7ff0a8d88665f4e557566e7b412e75edee2627fe
2015-10-06 18:49:49 -07:00
Aaron Schulz
beb385caf6 Remove ?maxlag check in index.php
Any "good citizen" bots should be using the API
anyway and less good bots will probably not bother
sending the parameter

Change-Id: I38f45b397d29b4709a932afc6319a1323e8d1200
2015-10-03 13:16:03 -07:00
Aaron Schulz
0a1c04beae Added $wgDataCenterId/$wgDataCenterRoles
* This is used to set sticky DC cookies to avoid
  session replication lag (which also makes sure
  ChronologyProtector works)

Bug: T91816
Change-Id: I7bc2f8185a3c05cb3ca5ccc42d300eccffae48e1
2015-10-01 00:52:54 -07:00
jenkins-bot
fcbf108445 Merge "Fixed some doc errors in tryNormaliseRedirect()" 2015-08-28 09:39:33 +00:00
Aaron Schulz
c1b47dd23e Fixed some doc errors in tryNormaliseRedirect()
Change-Id: I8f9397d05de1c0bae33497d1f9e3146939599380
2015-08-28 10:24:17 +02:00
jenkins-bot
e205809943 Merge "Move job running till after deferred updates" 2015-08-27 09:40:35 +00:00
Aaron Schulz
5781d54545 Fixed usage of ChronologyProtector in MediaWiki
* Placed an LB shutdown() call in dePreOutputCommit() so that the
  positions are properly included in sessions before session write().
  They need to be part of the synchronous updates that happen before
  the user gets the response, otherwise it defeats the whole point of
  the system.

Bug: T101224
Change-Id: Idf367c2aa9aae432a0c4d7cc697366aa544d77f2
2015-08-24 15:11:14 -07:00
Aaron Schulz
dee83538c4 Move job running till after deferred updates
* If $wgJobRunRate is > 0, such jobs should be run *after* all
  of the request-specific updates and job insertions. That way,
  some random job fatal does not interfere with the request
  updates.

Change-Id: I4bdb4e34404ce2d00997b545d681f3a0fb910d4f
2015-08-24 14:31:06 -07:00
Kunal Mehta
ad9f14d662 Support redirects in JavaScriptContent
When a JavaScript page is moved, a "redirect" in the form of
mw.loader.load(...) will be left behind, so any other
JavaScript loading the page that way will still work, albeit
with an extra HTTP request.

This also implements Content::getRedirectTarget(), so redirects
are marked properly in the database, and users viewing them
are redirected properly. A magic "/* #REDIRECT */" comment
must be in front of the mw.loader.load call. This is done so
that pages which currently are just one mw.loader.load call
aren't turned into redirects.

Bug: 71200
Bug: 33973
Change-Id: I10fdff087a901da56fad64531f0e382f90ebcf37
2015-07-20 15:36:49 +00:00
Timo Tijhof
155d555b83 MediaWiki.php: Redirect non-standard title urls to canonical
Urls that use the page's title and no extra query parameters now redirect
to the standard url format.

Previously we only did this for variations of the title value (e.g. "Foo%20Bar"),
not for variations of the overall url structure (like title=Foo -> /wiki/Foo).

Existing redirect (unchanged):
 /wiki/Foo%20Bar
 /w/index.php?title=Foo%20Bar

New redirects:
 /wiki/Foo_Bar?action=view
 /w/index.php?title=Foo_Bar
 /w/index.php?title=Foo_Bar&action=view

Any intentional (or unintentional) ways a url can be rewritten by the server,
such as "/?title=Foo_Bar" in case of Wikimedia, are redirected as well.

While this has been a problem for many years, it went unnoticed until
recently when Google started to index significantly more results of
the "/?title=<name>" form. This query returns "About 3,220,000 results":
https://google.com/search?q=site:en.wikipedia.org+inurl:title+-intitle:title

The only change in logic is that the titlekey comparison is now no longer a
factor in deciding whether to redirect. Instead the existing comparison for the
entire url is used to cover this.

However I kept titlekey comparison in the redirect-loop check as otherwise this
check would throw on all canonical page views where no redirect can be made.
Added a comment explaining how this redirect loop was possible.

Bug: T67402
Change-Id: I88ed3525141c765910e66188427b9aab36b958a9
2015-06-23 06:19:07 +00:00
Timo Tijhof
64431fcf8e MediaWiki.php: Factor out tryNormaliseRedirect
This is in preparation for fixing T67402, which requires adding
logic inside this condition block. However the to-be-added code
will influences whether or not a redirect should be made.

In case a redirect is not made, it has to fall through to the next
'elseif' handler in MediaWiki::performRequest(), which is not possible
from inside the 'if' block.

Hence, move it out in a separate block and use a boolean return value
to communicate whether the case has been handled.

This also allows us to unit test this thing. Which is desperately
needed. Albeit ugly as it requires lots of mocking.

Change-Id: If3157f2ff1fd3ab2ca20a5d1f550d864ea62c493
2015-06-23 07:18:35 +01:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
Timo Tijhof
f57b6af45b WebResponse: Implement statusHeader() using the new HttpStatus::header()
* Convert existing use of WebResponse::header() for HTTP status headers
  to use this new statusHeader() method.

* Extend unit test forFauxResponse.

I'm not calling HttpStatus::header directly in code. We keep the abstraction
layer of WebResponse so that responses can continue to be mocked/fauxed without
affecting the outer HTTP response.

Change-Id: I8a536e16659fa88b54cffa1457efb889efa5fcd6
2015-06-04 02:27:30 +00:00
Aaron Schulz
016fd0b776 Refactored entry points to have uniform shutdown handling
* Added doPreOutputCommit() and doPostOutputShutdown(),
  which most entry points just using the later
* Also fixed problem where text profiling did not show up
* Avoid calling triggerJobs() in the file streaming
  entry points

Bug: T100127
Bug: T100085
Change-Id: Ibc7e768fd483389a01847f08cdeba4058c853d3f
2015-05-29 20:40:46 +00:00
Brad Jorsch
f0b510f9d9 Pass a message key to MalformedTitleException constructor
'badtitletext' seems to be equivalent to the situation before I4da8ecb4,
so let's go with that.

Bug: T99818
Change-Id: Ifdf66d2806df3ae7d2d378343491dd8976c31981
2015-05-20 15:44:01 -04:00
Aaron Schulz
2a176864eb Made pushLazyJobs() handle all queue groups
* This avoids __destruct() warnings
* Also remove push() code in __destruct() that just made
  more warnings since it's too late

Change-Id: I32a3cfabc94e36b9d2808b45d55209f3df46e47d
2015-05-19 02:12:28 +00:00
Aaron Schulz
7e97ce6a44 Added JobQueueGroup::lazyPush method
* Made use of this in triggerOpportunisticLinksUpdate()
* This will defer and better batch job insertion
* Lazy job insertion and other deferred updates
  make use of register_postsend_function if present
* Also cleaned up some return types and exceptions
  in JobQueueGroup

Bug: T99302
Change-Id: I3a3968d75cb37563f970be08e63f31a090e0e037
2015-05-18 16:26:59 +00:00
jenkins-bot
e788bf54c7 Merge "Provide detailed information about invalid titles" 2015-05-04 17:26:03 +00:00
Bartosz Dziewoński
708918ccbf Provide detailed information about invalid titles
includes/Title.php
* The private method Title::secureAndSplit() now throws a
  MalformedTitleException instead of returning false on invalid
  titles.
* Added Title::newFromTextThrow(), which behaves exactly like
  Title::newFromText() but throws MalformedTitleException instead of
  returning null on invalid titles.

includes/title/MediaWikiTitleCodec.php
* Provide more information with the thrown MalformedTitleExceptions.

includes/MediaWiki.php
* Use the new Title::newFromTextThrow() to get detailed error
  information, display it.

Change-Id: I4da8ecb457a77473e32d745ba48ab8505b35e45f
2015-04-30 23:50:07 +02:00
Aaron Schulz
7ea13643f5 Added $wgTrxProfilerLimits and slow query limits
* Limits are now configurable instead of being hard-coded

Change-Id: I99133586eb82e8e9e84061548c8d1a99695fde5c
2015-04-28 10:18:11 +02:00