Commit graph

385 commits

Author SHA1 Message Date
umherirrender
8ad1c92441 Fixed some @params documentation (includes/*)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I7f605aa9e117b5fd80d9b1440864fe526d2b14a5
2014-04-20 21:16:57 +02:00
jenkins-bot
996eed90e7 Merge "Avoid site outages due to broken job queues" 2014-04-15 01:13:29 +00:00
Aaron Schulz
637297c7c5 Commit DB changes as normal on exception-based GUI errors
Bug: 62091
Change-Id: I36cd645d5f7a939b60f306e5fc093507693b097b
2014-04-14 16:51:55 +00:00
Aaron Schulz
2eac3278a4 Avoid site outages due to broken job queues
Change-Id: I71f1bef886850679812e23b8a49fc9b80d9b6102
2014-04-11 15:00:12 -07:00
Aaron Schulz
4865e34630 Avoid making a derivative request to Special:RunJobs when the job queue is empty
Checking that the job queue is nonempty is cheap, and when the result is
negative (that is, when the queue is actually empty), it spares from having to
launch a derivative request to Special:RunJobs for job processing. The
derivative request is not cheap, because processing it requires having an
additional full instance of the application. This patch adds the check.

Bug: 60210
Change-Id: Icb95c35f4a8a3c9d4b5aff97fdfabf596d05940a
2014-04-09 22:49:46 -07:00
Jesús Martínez Novo
fab0150973 Added $wgRunJobsAsync to allow running jobs the old way
Adding a new global variable (DefaultSettings.php) named $wgRunJobsAsync, that
defaults to true, to allow installations to force the old synchronous job
execution on page requests instead of the new asynchonous one.

The asynchronous job queue execution was added in 1.22, executing a new shell
script, and it caused major problems for third party installations with restricted
environments or limited system resources, and also because a lot of bugs that went
unnoticed because WMF wikis do not use the on-request job queue.

In 1.23 it was changed to use a new internal HTTP request to a MediaWiki entry point
to perform the request, While some of the bugs were solved, it could still have
performance problems for opening a new process and loading a lot of MediaWiki PHP
objects on it, just to perform a request to see if there are pending jobs and execute
them (it doesn't check if there are jobs to execute). While this may improve speed
on single page requests, because it does not block the execution of the current page,
it has't been thoroughly tested on third party installations. And what is more
important, now there's no way to revert back to the old way of handling those jobs,
as it was done in 1.22 (setting $wgPhpCli = false). This means, in case of major
bugs on the job queue execution due to the new request, there's no way to fix it
other than editing MediaWiki PHP files directly, or completely disable the
on-request job queue handling (some hosts can't set up a cron job for that)

This situation makes it critical to have a safe way to revert to the old job queue
handling, at least until testing that the current implementation doesn't break a lot
of installations.

Bug: 61387
Change-Id: I892a3608228ec0a1c63268fb6180ec679c315980
2014-04-09 21:25:59 +02:00
Erik Bernhardson
6f65ab9a22 Reduce connect timeout in MediaWiki::triggerJobs
The fifth parameter to fsockopen is the number of seconds
to wait before declaring the connection a failure. When not
provided it defaults to the php ini value of 'default_socket_timeout'
which is 60s.  Under no circumstances should we wait 60s to
connect to ourselves,  100ms seems like a reasonable timeout
since its explicitly connecting to itself, but the exact timeout
could be slightly longer.

Change-Id: I18d328274ddf1e0848fce40ebf39e0466b5c4d5d
2014-03-25 15:31:56 -07:00
Aaron Schulz
96528b1f88 Avoid header notice log spam from RunJobs API
* Moved ApiRunJobs to a special page instead of going through
  ApiMain and having to fight the logic there. As a separate
  internal API, this does not show up on the API help page and
  is no longer effected by $wgEnableAPI.

bug: 62233
Change-Id: I1db6f526d02e130a66ee03289858a734d89e6c00
2014-03-18 22:30:50 +00:00
Aaron Schulz
dc8283a386 Call ignore_user_abort() right before committing to the DB
Change-Id: I48b9e90418c38c7c9970cf0739a877b5be3a2b64
2014-03-12 14:05:33 -07:00
Nik Everett
142db358fc Have ApiRunJobs run periodic tasks
Without this apirunjobs can't handle delayed jobs.

Small inline comment updates as well.

Change-Id: I85ba0ac99eaebc16eaa7e2eb7019546241f3149c
2014-03-07 10:40:38 -05:00
Aaron Schulz
df2f5dc7a6 Avoid API error/header leakage from jobs API
* Also cleaned up some HTTP headers while at it

bug: 62092
Change-Id: Ia2c1643e47aae53916c107c901cca654dc438a16
2014-02-28 21:44:49 -08:00
Aaron Schulz
1ec4e48b83 Handle some extra cases in triggerJobs()
* $wgEnableAPI may be false, though it is a basic part of MediaWiki.
  This only disables api.php and not internal callers so just do that.
* Various socket functions may disabled. Though this is a terrible
  wack-a-mole security measure to handle MediaWiki exploits or
  non-authenticated services, it may still be in use, so work
  around it.

Change-Id: I54a209e003ffce1fe36a5898fe0da735cf5e32e4
2014-02-28 05:52:13 +00:00
Aaron Schulz
2ea4d7ae8c Moved job running via $wgJobRunRate to a special API
* A background internal HTTP request used to load the API.
  This handles job execution and related periodic tasks.
  It avoids blocking the main request or breaking it on fatals.
* This method avoids problems that shelling could have:
** Mismatched apache/cli PHP config or versions.
** Had to start a new process and could not use opcode cache.
** safe_mode and open_basedir restrictions required a fallback.
** Some wiki farms may not respect the --wiki parameter.
** wgMaxShellMemory applies to spawned PHP processes.
** Spawning processes is more prone to DOS due to a lack
   of proper limiting/pooling.

bug: 60208
bug: 60210
bug: 60698
bug: 60844
Change-Id: I78b0e709301ac4a0c7b7ed337d6969d7546674bf
2014-02-27 19:12:04 +00:00
Aaron Schulz
e14bb583a4 Added --wiki parameter to runJobs.php call
Change-Id: I8f729263397bedf1c293b14704798e7a4ec60581
2014-02-07 11:19:41 +00:00
Aaron Schulz
96b678e110 Various fixes to job running code in Wiki.php
* Make it async on Windows
* Fixed possible "invalid filename" errors on Windows
* Redirect output to dev/null to avoid hanging PHP

bug: 60231
bug: 58719
Change-Id: If4e52b86fb3a1bff283dfba3dfac213367f47543
2014-01-29 22:56:38 +00:00
Siebrand Mazeland
13b3fe2981 Remove deprecated MediaWiki::articleFromTitle() and WikiPage::createUpdates()
No uses found in core or Gerrit extensions.

* MediaWiki::articleFromTitle() (deprecated in 1.18)
* WikiPage::createUpdates() (deprecated in 1.18)

Change-Id: I689249b579ee89f883b97fdb4ac79fb3502c45dc
2014-01-22 18:28:36 +01:00
jenkins-bot
bf8a10be13 Merge "Don't convert bad curids to the Main Page" 2014-01-04 00:12:12 +00:00
jenkins-bot
91e4128955 Merge "Add fastcgi_finish_request where appropriate" 2014-01-03 16:06:47 +00:00
Tyler Anthony Romeo
6aeb565733 Add fastcgi_finish_request where appropriate
When using PHP-FPM to run PHP on a web server, the function
fastcgi_finish_request is provided, which flushes all response
data to the client and closes the connection, allowing PHP
to perform additional work without delaying the client.

This adds fastcgi_finish_request calls in index.php and api.php
where appropriate and if the function exists, so that the connection
is closed once the output is definitely finished.

Change-Id: Ic67a8956545874e94df5198088c0f4aa58ca376e
2014-01-03 04:02:52 -05:00
Jackmcbarn
1ad293efff Don't convert bad curids to the Main Page
When a curid that doesn't exist is requested, cause the "Bad title" error
to be displayed rather than displaying the Main Page.

Bug: 58885
Change-Id: If54ca35b3799b8cf2f76e7f8a03001caa1fac676
2014-01-02 19:56:47 -05:00
umherirrender
65a4ae9fe9 Change Title::getInterwiki() in conditions to Title::isExternal()
Change-Id: Icce26e6194ae96f262029554e05b49117d5e112e
2014-01-02 11:59:10 +01:00
umherirrender
0bc583af2c Move closing parenthesis from multi line if and function to own line
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.

Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
2013-12-01 21:39:00 +01:00
Tim Starling
8af1c9503d Introduce WebRequest::getProtocol()
The use of static server detection outside of its intended use case
(i.e. at the start of DefaultSettings.php), for example in r93258, was
an architectural error. Every other bit of information about the web
request in non-setup code comes from non-static methods of WebRequest,
which allows the request object to be meaningfully replaced or
subclassed. The situation became increasingly ridiculous as more
callers of WebRequest::detectProtocol() were introduced. Two of the
callers were calling it non-statically! I suppose they had the right
idea, in a way.

Using a non-static call allows caching, which is a nice additional
benefit.

WebRequest::detectProtocolAndStdPort() was introduced in r93258 as part of
the introduction of WebRequest::detectProtocol(). It was basically
useless. Grep indicates there was only one caller in core and WMF
deployed extensions, and it is patched here.

Change-Id: Ia0a61e98fbff7a46ceaeebcb02236e5eac3df0e1
2013-11-06 21:13:10 +04:00
Chad Horohoe
2f28d7dd70 Kill $wgArticle
This has been deprecated since 1.18, throwing warnings for over a
year and a half now, and nothing uses it anywhere in core or deployed
extensions.

No extensions in SVN seem to have this anymore (we must've cleaned
up well), and the only affected extensions in Git are:

- BlueSpiceExtensions
- DynamicPageList (commented, so shouldn't be a worry)
- SemanticPageMaker
- TwitterCards

If you're the author of one of these extensions or are feeling like
cleaning up ancient code, please get them fixed.

Change-Id: I8f3a54c14be4b38a8b917e32ba3a832e0a7ca116
2013-11-01 14:17:26 -07:00
Timo Tijhof
4468a46af2 exception: Use MWExceptionHandler::logException in more places
Most code replaced wasn't exactly like what logException does
but most probably should be.

A few implementation differences with the code it replaced in
various places:

* MWException if-guards
  Was there only to prevent a crash because getLogMessage is an
  MWException method. Now that logException is generic, it seems
  sensible to start logging those as well (follows-up a97f3550a0).

* Exception::getTraceAsString
  Now using MWExceptionHandler::formatRedactedTrace instead.
  It wasn't using it because that method didn't exist yet.

Notes:

* DatabaseError::getLogMessage
  Removed as this override was no longer doing anything (we're using
  MWExceptionHandler::getLogMessage instead of $e->getLogMessage).
  Introduced isLoggable() to take over the responsibility of indicating
  when an exception should not be logged (follows-up bcb9f9e1c0).

* DeferredUpdates and Wiki.php
  Both specificy MWException. Though ApiMain intends to catch all
  and only logged MWException because it couldn't otherwise, these
  actually only catch MWException (as opposed to catching all and
  having an if-statement inside). Left those as-is to have them
  continue propagate other exceptions.

* JobQueueFederated and JobQueueGroup
  All specify to catch JobQueueError only.
  Not sure whether it should catch other exceptions. It now can,
  but I'll leave it as is in case it intends to have those be
  handled elsewhere (or fatal).

Change-Id: I4578a0fe7d95a080f1a3b292ce7ae73a4d5fcaca
2013-10-30 15:46:35 -07:00
Bartosz Dziewoński
dfc3e3df90 Correctly update wl_notificationtimestamp when viewing old revisions
== Prelude ==
wl_notificationtimestamp controls sending the user e-mail
notifications about changes to pages, as well as showing the "updated
since last visit" markers on history pages, recent changes and
watchlist.

== The bug ==
Previously, on every view of a page, the notification timestamp was
cleared, regardless of whether the user as actually viewing the latest
revision. When viewing a diff, however, the timestamp was cleared only
if one of the revisions being compared was the latest one of its page.

The same behavior applied to talk page message indicators (which are
actually stored sepately to cater to anonymous users).

This was inconsistent and surprising when one was attempting to, say,
go through the 50 new posts to a discussion page in a peacemeal
fashion.

== The fix ==
If the revision being viewed is the latest (or can't be determined),
the timestamp is cleared as previously, as this is necessary to
reenable e-mail notifications for given user and page.

If the revision isn't the latest, the timestamp is updated to
revision's timestamp plus one second. This uses up to two simple
(selectField) indexed queries per page view, only fired when we
do not already know we're looking at the latest version.

Talk page indicator is updated to point at the next revision after the
one being viewed, or cleared if viewing the latest revision. The
UserClearNewTalkNotification hook gained $oldid as the second argument
(a backwards-compatible change). In Skin, we no longer ignore the
indicator being present if we're viewing the talk page, as it might
still be valid.

== The bonus ==
Comments and formatting was updated in a few places, including
tables.sql and Wiki.php.

The following functions gained a second, optional $oldid parameter
(holy indirection, Batman!):
* WikiPage#doViewUpdates()
* User#clearNotification()
* WatchedItem#resetNotificationTimestamp()

DifferenceEngine gained a public method mapDiffPrevNext() used
to parse the ids from URL parameters like oldid=12345&diff=prev,
factored out of loadRevisionIds(). A bug where the NewDifferenceEngine
hook would not be called in some cases, dating back to its
introduction in r45518, was fixed in the process.

Bug: 41759
Change-Id: I4144ba1987b8d7a7e8b24f4f067eedac2ae44459
2013-10-27 17:47:53 +00:00
csteipp
0adbbacb0a Move forceHTTPS check until after wgTitle is setup
Since the forceHTTPS logic checks if the user is logged in, which calls
all sorts of hooks, some that assume that $wgTitle is setup (like
AbuseFilter for preventing autocreation of usernames), move the
logic down a few lines.

Bug: 55435
Bug: 53498
Change-Id: I53c82921595a9014b555e4ec468e5ba10454cd3a
2013-10-07 12:47:43 -07:00
jenkins-bot
722d9a971b Merge "Add Vary: X-Forwarded-Proto for https redirects" 2013-10-01 04:46:01 +00:00
Niklas Laxström
3e72e1df6a Move 'main page as default title' further down to unbreak diff urls
The feature that oldid and diff parameters override the title was
not working if wiki's main page is a special page, because that
behaviour was excluded for special pages. Hence moved the main
page as default title code further down, so that we will hit the
no title provided condition for the oldid and diff parameter only
links.

Bug: 54452
Change-Id: I62cc6e2568a4db988fcf113955ba02710e42a1d2
2013-09-30 23:17:09 +00:00
csteipp
d5513c90cd Add Vary: X-Forwarded-Proto for https redirects
From mark's comments on https://gerrit.wikimedia.org/r/#/c/86268/, we
should be setting Vary: X-Forwarded-Proto when the response varies on
the protocol, which is the case for redirects from http->https.

Change-Id: Ie6dc9da4e804199b99e3a7abbea7775a181801a0
2013-09-30 11:04:45 -07:00
aude
2e8230a45c Pass context to Action::factory
Fixes a PHP Notice that was introduced on every page (follows-up 01d81d88)

Bug: 54344
Change-Id: I24f01c7244089e73b0cf2e0af37c365f9dd051b2
2013-09-19 22:45:56 +00:00
csteipp
5e5843689a Also redirect if prefixed https cookie is preset
After we switch to not using prefixes, still respect prefixed cookies
for users who are currently logged in.

Bug: 53538
Change-Id: I950e36e10420209fca28c41c438c363a3f332c14
2013-08-30 17:00:03 -07:00
csteipp
188823c074 Remove prefix from forceHTTPS cookie
Match CentralAuth, and set appropriate expiration too.

Bug: 53538
Change-Id: I3c16ff92781e1a72346058ae3838d8fc47019d55
2013-08-29 13:40:07 -07:00
Tim Starling
f0b346d6d0 Core support for disabling HTTPS based on GeoIP
* Introduce a hook allowing automatic redirects to HTTPS to be
  disabled on the basis of client IP address.
* Make User::requiresHTTPS() return false if the client IP is
  blacklisted as such.
* On login, make the "stick HTTPS" option default to false if the
  client IP address is blacklisted as such.
* Do not redirect anonymous requests to HTTPS.
* If $wgSecureLogin is enabled, link to the HTTPS login page *via*
  the HTTP redirect, so that there is no need to vary the cache of
  anonymous page view HTML on client IP address.

Change-Id: Iaa9dd2108431b8c35e05db4bfe78a629018a003c
2013-08-21 13:43:46 +10:00
Chad Horohoe
3c2b7246f1 Fix super long line
Change-Id: Ib76a2b6209137bcf18db9b756fa7406547611129
2013-08-19 18:16:10 -07:00
Chad Horohoe
d95454b44f Add some logging for weird redirected POSTs
Change-Id: Ib19ebc2306e85fdb8c29b5850f2306bec7907938
2013-08-19 17:29:50 -07:00
Tyler Anthony Romeo
374a0ad943 Change secure login to use a user preference
Removed the wpStickHTTPS checkbox from the login form, and instead
just use the user's preferences along with whether they came from
HTTPS or not to determine if they should stay in HTTPS.

Bug: 29898
Bug: 52283
Change-Id: I69e9cb23b8d700e821b8a961c672958e4e19e4f8
2013-08-19 16:57:04 -07:00
Aaron Schulz
d4877cc3ad Hide exceptions in MediaWiki::doJobs() as deferred updates do.
Change-Id: Ia83029d4c06bf3f2be6ad0797e815464e44fd088
2013-05-14 18:36:07 -07:00
umherirrender
c0d61543b2 Pass array to Title::getLocalURL instead of an urlencoded string
This makes code easier to read and the urlencode is done inside
getLocalURL or friends

Change-Id: I21b988890356d11835eedba12a90a347bf0905b2
2013-04-27 08:22:01 +00:00
Aaron Schulz
ccabd0efb0 Changed on-request job running to shell out instead of doing a loop.
* Also factored out a new wfShellExecDisabled() function.
* This will keep the process in the background if possible to avoid
  killing site performance, especially with slow jobs.
* This also keep fatals and uncatcheable exceptions from
  hitting the user.
* If $wgPhpCli is not set to an actual path or safe mode
  is on, then the old code will be used.

Change-Id: I6a28152251659ee53eee2604f16d5bf02c85a44f
2013-04-25 23:46:05 +00:00
umherirrender
ef2f507d23 Fixed spacing in files direct in includes folder
Added spaces before if, foreach
Added some braces for one line statements

Change-Id: Ibb8dd102db045522d12ff939075ba7420d95ab6b
2013-04-21 06:38:49 +00:00
Aaron Schulz
110450d73c Wrap job execution around profiling calls.
Change-Id: Idf184b7c2f7003032d3a28830ee9c4a1eaeaa98b
2013-04-18 12:22:33 -07:00
Alexandre Emsenhuber
cd4dd7ca85 Correct case of Title::getPrefixedDBkey() calls
Change-Id: Ic9ef0cc6dbb0a8eb0ee93432299ef59759223e2c
2013-03-09 21:14:22 +01:00
umherirrender
d63121016d fix some spacing
Added/removed spaces around logical/arithmetic operator
Reduced multiple empty lines to one empty line
Removed wrong tabs before comments at end of line
Removed too many spaces in assigments

Change-Id: I2bba4e72f9b5f88c53324d7b70e6042f1aad8f6b
2013-03-07 17:53:21 +01:00
umherirrender
6fbbbd17ca fix some spacing
Change-Id: Ie7bb35871cc99237f3a655f7db22ca1f0646df5e
2013-01-27 14:21:50 +01:00
Aaron Schulz
6eef765b85 [JobQueue] Optimized JobQueueGroup::pop().
* This also simplifies MediaWiki::doJobs().

Change-Id: I58ee2313453c64d4e8d91f3a65181aaa9c2e847f
2013-01-14 16:40:27 -08:00
Brian Wolff
9e5802ef66 Reuse the WikiPage object to save a db query
Early in the request we call Action::getActionName to get the action
name for HTMLFileCache to see if the current action is cached.
This causes data to be loaded from the WikiPage object
stored in the main request context. However, later when
we're initializing the Article object, we overwrite that
WikiPage object with the one used by the Article object.
Later on the WikiPage object has to be re-loaded, and
hence the exact same db request is run twice in one request,
which seems wasteful.

Instead, initialize the Article object using the WikiPage
object from the context, in order to save the already
loaded data.

Change-Id: I292f0d70feb505fae5fa955fd735d85ad3b22fea
2013-01-04 13:05:09 +00:00
Niklas Laxström
bda755617e Rewritten $wgAdaptiveMessageCache
This was developed for translatewiki.net, which has but
millions of messages in MediaWiki namespace. To avoid
exploding the message cache, not all of them can be
loaded. Instead of logging what messages have been
requested, now it uses the fact that if the
message has not been customised in the
site language, there is most likely
no reason to load it in any other
language either.

Change-Id: I6dd81b3858acfd2b73332e46668d46015e99c748
2012-10-26 12:48:52 -07:00
Tim Starling
6cb1fe1fe4 Fixed the caching of redirects in Squid
Was accidentally broken in Ie38ae198, where the request URL was compared
against the final target title, not the title of the redirect. Also
tested squid purges of redirects, since that code presumably hasn't been
tested for a while, and committed my very advanced testing framework.

Change-Id: Ib2ffe8e109bcc4859e22df5470bddfdb09e67afa
2012-10-17 11:00:07 +11:00
ASchulz
b567f3602e [JobQueue] Job queue refactoring and generalizing.
* Added support for different queue types and methods for storing queues.
* Treat each job type as being on its own queue, at least logically.
* Added $wgJobTypeConf to configure queue types for each job type.
* Improved the job DB table so that duplicate job checks actually work
  and are faster. Also improved the method for popping rows of the table.
* Disabled duplicate job removal for everything except refreshLinks.
  The DELETE statements just add DB overhead and are not useful for cheap
  jobs, especially ones with start/end params (which are unlikely to have
  exact duplicates).

Change-Id: I49824c7fa855fea4ddcac5c9901ece8c2c0101d0
2012-10-16 09:39:58 +11:00