This adds a method to LinkFilter to build the query conditions necessary
to properly use it, and adjusts code to use it.
This also takes the opportunity to clean up the calculation of el_index:
IPs are handled more sensibly and IDNs are canonicalized.
Also weird edge cases for invalid hosts like "http://.example.com" and
corresponding searches like "http://*..example.com" are now handled more
regularly instead of being treated as if the extra dot were omitted,
while explicit specification of the DNS root like "http://example.com./"
is canonicalized to the usual implicit specification.
Note that this patch will break link searches for links where the host
is an IP or IDN until refreshExternallinksIndex.php is run.
Bug: T59176
Bug: T130482
Change-Id: I84d224ef23de22dfe179009ec3a11fd0e4b5f56d
Domain is an important property to document for callers. For example,
random numbers are often used in calculations that are input into array
index calculations and the knowledge that a function can or cannot ever
return the integer 1 helps avoid rare off-by-one errors that may occur.
`int( wfRandom() * count( $array ) )` will always yield an in-bounds
index if wfRandom() returns [0, 1) but can make no such guarantee for
[0, 1].
It's not immediately obvious from the implementation whether the
endpoints of the domain of wfRandom() are inclusive or exclusive. This
patch calculates the minimum and maximum results and documents it.
For its minimal value, given `mt_getrandmax()` returns 1 and `mt_rand()`
returns 0:
$max = mt_getrandmax() + 1;
$max = 2;
$rand = ( mt_rand() * $max + mt_rand() ) / $max / $max;
$rand = ( 0 * 2 + 0 ) / 2 / 2;
$rand = 0;
For its maximal value, given `mt_getrandmax()` returns 2^31 - 1 and
`mt_rand()` also returns 2^31 - 1.
$max = mt_getrandmax() + 1;
$max = 2^31 - 1 + 1;
$max = 2^31;
$rand = ( mt_rand() * $max + mt_rand() ) / $max / $max;
$rand = ( (2^31 - 1) * 2^31 + 2^31 - 1 ) / 2^31 / 2^31;
$rand = ( 2^62 - 2^31 + 2^31 - 1 ) / 2^31 / 2^31;
$rand = 2^62 / 2^62 - 1 / 2^62;
$rand = 1 - 2^-62; // Less than 1.
Change-Id: Ib179d70902e231eaeeafe6449f505464eb25204d
For maintenance scripts it is usually harmful to throw an exception.
For jobs the exception was already caught and handled appropriately,
so this can continue as before. For DeferredUpdates it was extremely
harmful to throw an exception. So in the web case, reduce the timeout to
1s and continue as normal if the 1s timeout is reached. This allows the
DeferredUpdate to be throttled without being killed.
In the updater, increase the replication wait timeout to 5 minutes.
ALTER TABLE could indeed cause replication lag, but exiting the update
script with an exception will probably ruin your day. Update actions are
not necessarily efficiently restartable.
Do not call JobQueue::waitForBackups() when jobs are popped. Maybe it
makes sense to call a queue-specific replication wait function for
bulk inserts, like copyJobQueue.php, but doing it when jobs are popped
just makes no sense. Surely the worst that could happen is that the
queue would become locally empty? Removing this waitForBackups() call
avoids waiting for replication twice when JobQueueDB is used.
Bug: T201482
Change-Id: Ia820196caccf9c95007aea12175faf809800f084
And include tests :)
This code is independent of MediaWiki, but not really large enough to be
worth librarizing right now.
Bug: T200626
Change-Id: I022c074e8a708fb5219bc4ff4b53e7e31f60dc4b
The former is already a wrapper around ObjectCache::getInstance().
The latter was identical to ObjectCache::getLocalClusterInstance().
Bug: T115890
Change-Id: Ib4e43bc8d3f4ac9f7a453e36dcce9b3d962666ba
Now that all our supported PHP versions have array_filter()
with a third parameter, these functions aren't needed anymore.
Depends-On: I3b097a1a048baabcaca15dc214a3a1bb06e746cc
Depends-On: I0187e27ac47cbab099249572201d1a649226a734
Change-Id: I7cabd0252691a083cb749cf9d3a7a23f1d076c39
If the request URL was not normalized, for example having a double slash
in it, this could cause it to fail to match in the PathRouter. But the
canonicalizing redirect was using the normalized URL, causing a redirect
loop exception.
So:
* If the PathRouter fails to match with the original URL, try matching
against the normalized URL. This allows it to still work for
normalized URLs with a double slash in the title part of the path.
* Have WebRequest::getFullRequestURL() always return the URL without
removing dot segments or interpreting double slashes. Just append
the path to the server.
* Make MediaWikiTest.php use WebRequest instead of FauxRequest, allowing
it to reproduce the exception in question. Add relevant test.
* Add tests for the new PathRouter behaviour.
Bug: T100782
Change-Id: Ic0f3a0060904abc364f75dae920480b81175d52f
This fixes 26 of the phan-taint-check warnings on MW core. Some
are outright fixed, others are false positives that were suppressed.
This really only covers some of the easy ones. There are still
314 warnings to go.
Change-Id: I30463bc3a09fd4324d190de8533f51784764dd3a
When expanding a URL, don't overwrite an explicitly specified port or
add a port to a foreign URL. $wgHttpsPort is only useful for a very
specific case: when $wgServer is protocol-relative and HTTPS is
requested.
Documented correct use of $wgHttpsPort in DefaultSettings.php. Fixed
invalid "@see", in Doxygen it can only point to "classes, functions,
methods, variables, files or URL".
Added test cases which previously failed.
Change-Id: Id65c58300d22712212b6605711ff916916e8768b
$params is a variadic parameter now, so it must be treated as an
array. Same thing for wfMessageFallback().
Change-Id: I606deb7ea0fb9be25ac79aadfbab0cc44bdc36a0
Uses new PHP 5.6 syntax like ...parameter unpacking and
calling anything looking like a callback to make the code more readable.
There are much more occurrences but this commit is intentionally limited
to an easily reviewable size.
Change-Id: Idcec077ef3fdf029b632cceafd0150851ad723e3
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '
(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)
Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).
Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
`$a <=> $b` returns `-1` if `$a` is lesser, `1` if `$b` is lesser,
and `0` if they are equal, which are exactly the values 'sort()'
callbacks are supposed to return.
It also enables the neat idiom `$a[x] <=> $b[x] ?: $a[y] <=> $b[y]`
to sort arrays of objects first by 'x', and by 'y' if they are equal.
* Replace a common pattern like `return $a < $b ? -1 : 1` with the
new operator (and similar patterns with the variables, the numbers
or the comparison inverted). Some of the uses were previously not
correctly handling the variables being equal; this is now
automatically fixed.
* Also replace `return $a - $b`, which is equivalent to `return
$a <=> $b` if both variables are integers but less intuitive.
* (Do not replace `return strcmp( $a, $b )`. It is also equivalent
when both variables are strings, but if any of the variables is not,
'strcmp()' converts it to a string before comparison, which could
give different results than '<=>', so changing this would require
careful review and isn't worth it.)
* Also replace `return $a > $b`, which presumably sort of works most
of the time (returns `1` if `$b` is lesser, and `0` if they are
equal or `$a` is lesser) but is erroneous.
Change-Id: I19a3d2fc8fcdb208c10330bd7a42c4e05d7f5cf3
* Fix ResourceLoaderClientHtml to return what it was documented to
return, a WrappedStringList. It accidentally used the wrong join()
method, causing it to create a plain string too early.
* Update method documentations from ClientHtml::getBodyHtml to
BaseTemplate::getTrail for 'bottomscripts', and from
ResourceLoader::makeInlineScript for 'reporttime'.
* Update BaseTemplate::getTrail to join by new line instead of native string
concatenation.
This by itself would suffice for the most common case, but in order
to also account for the possibility of extensions using hooks for
'SkinAfterBottomScripts' that concatenate, update bottomScripts()
to pass a plain string to the hook and merge it later.
Change-Id: If0d227cb9db67229a27d489c082db790ea8e3840
Disable it in specific files and places where there are legitimate uses
to access $_GET and $_POST directly.
For EditPage, which wants to output $_POST for debugging information,
introduce WebRequest::getPostValues() as a wrapper, matching the
existing ::getQueryValues().
Change-Id: I2cb0a7012fb7ed29dcd720056b42f56508ddc5fa
The primary goal here is a defense in depth measure to
stop an attacker who found a bug in the parser allowing
them to insert malicious attributes.
This wouldn't stop someone who could insert a full
script tag (since at current it can't distinguish between
malicious and legit user js). It also would not prevent
DOM-based or reflected XSS for anons, as the nonce value
is guessable for anons when receiving a response cached
by varnish. However, the limited protection of just stopping
stored XSS where the attacker only has control of attributes,
is still a big win in my opinion. (But it wouldn't prevent
someone who has that type of xss from abusing things like
data-ooui attribute).
This will likely break many gadgets. Its expected that any
sort of rollout on Wikimedia will be done very slowly, with
lots of testing and the report-only option to begin with.
This is behind feature flags that are off by default, so
merging this patch should not cause any change in default
behaviour.
This may break some extensions (The most obvious one
is charinsert (See fe648d41005), but will probably need
some testing in report-only mode to see if anything else breaks)
This uses the unsafe-eval option of CSP, in order to
support RL's local storage thingy. For better security,
we may want to remove some of the sillier uses of eval
(e.g. jquery.ui.datepicker.js).
For more info, see spec: https://www.w3.org/TR/CSP2/
Additionally see:
https://www.mediawiki.org/wiki/Requests_for_comment/Content-Security-Policy
Bug: T135963
Change-Id: I80f6f469ba4c0b608385483457df96ccb7429ae5
