Commit graph

212 commits

Author SHA1 Message Date
dcausse
353ae71963 Extracted some code from PrefixSearch for re-usability by other search engines.
(This patch is needed for Ida9b9f8)

Bug: T112028
Change-Id: I35aece88333a65f6b1f55f7a87e2d14de4f5bea7
2015-12-02 17:18:08 +00:00
Reedy
5d032c0f65 Rename wfBaseConvert.php maintenance script
Change-Id: I3174ceb040e9029ad8049a3e49c119bfac4aa386
2015-11-24 23:34:21 +00:00
jenkins-bot
8732532d47 Merge "Implement HTMLComboboxField" 2015-11-24 19:20:07 +00:00
jenkins-bot
899bd46cbb Merge "Add script to fix content model of JSON pages" 2015-11-24 18:53:43 +00:00
Kunal Mehta
2986d47c90 Add script to fix content model of JSON pages
MediaWiki:Foo.json and User:Foo/bar.json pages now have a default
content model of JSON, but existing pages using those names will be set
to defaults of wikitext.

The content models of those pages are now set to "json", unless it has
invalid syntax, in which case it will be set to "wikitext".

For convenience, the script is automatically run as part of update.php.

Bug: T108663
Change-Id: I1412937ccea8e65dba58580beec79cbf2286ae01
2015-11-24 10:08:44 -08:00
jenkins-bot
aa5e579028 Merge "Move details of import logs into log params" 2015-11-24 08:52:12 +00:00
Brad Jorsch
42e257a1a2 API: Remove dbg, txt, and yaml formats
Per RFC: https://www.mediawiki.org/wiki/Requests_for_comment/Ditch_crappy_API_formats

Bug: T95715
Change-Id: I8e4e9ad3c96504cf4587cb4632504eff21b5ff75
2015-11-18 09:19:51 -05:00
Bartosz Dziewoński
77d759aab4 Implement HTMLComboboxField
It's a dropdown select with the ability to add custom options, or a
text field with input suggestions, whichever you prefer.

This is meant to be a replacement for HTMLSelectOrOtherField and
HTMLAutoCompleteSelectField.

In regular HTML mode, it uses HTML5 `<datalist>` element with no
custom JavaScript. This is supported by a wide range of browsers
(IE 10+, modern Firefox and Chrome, Opera 12+).

In OOUI mode, it uses a ComboBoxInputWidget.

Depends on: I14b40884f185fb4e5

Bug: T118119
Change-Id: I954d3d24ed4efe90be9596a1bd9586ba3aee1e23
2015-11-12 09:48:27 +01:00
umherirrender
dc78d4d12c Move details of import logs into log params
The detail information about the count of revision and the interwiki
title get moved into log params. To show the new information an
ImportLogFormatter was written, which changes the key for new log items.

This allows to show the detail information in the user language.
It also decouple the user supplied data from detail information

Change-Id: Iaa57da0fc3e20c33c94fd850cd02db96fdb32dac
2015-11-11 18:34:18 +01:00
Timo Tijhof
304eaf2868 objectcache: Reduce MediaWiki-specificness from Memcached classes
Follows-up f36b73e96c, which moved these classes to libs/objectcache.

* Fix wrong @throws in MemcachedPeclBagOStuff.
* Fix wrong @returns in MemcachedBagOStuff::getClient().
* Rename MWMemcached to MemcachedClient.
* Remove mention of 'memcached.php', which doesn't exist anymore.

Change-Id: I34dbc859be4778cea489fd2344f233f30452605c
2015-11-10 03:32:58 +00:00
aude
b0c81a3e09 Remove deprecated SiteObject and SiteArray classes
SiteObject and SiteArray were deprecated in 1.21 and
can't find any use of them anywhere else in gerrit.

Change-Id: Iff3ba8a60ac9566998cce828c4034066fdefe804
2015-11-04 13:43:32 +01:00
Ori Livneh
8340400f27 Add Timing interface
* Add `Timing`, an interface which mimics the W3C User Timing API.
  It provides a canonical way to store and retrieve markers (timestamps)
  and measures (timestamps + duration).
* As the initial use-case, use it to record 'requestShutdown'.

Change-Id: I36b29162ffcc091406df025463b0e2797e52f19a
2015-11-03 15:05:37 -08:00
jenkins-bot
41f9c91545 Merge "Add script to find orphaned LocalRepo files" 2015-11-03 10:48:31 +00:00
Reedy
d9d22e8f0b Remove ORM code from core
Bug: T114538
Change-Id: Ic4196ca9da927fc5c85b01cfff65f0636e3202ae
2015-10-29 13:06:36 +00:00
Aaron Schulz
828b598911 Add script to find orphaned LocalRepo files
Bug: T111838
Change-Id: I60ac7f3451c8240a0716933dcfff413669458206
2015-10-29 01:59:10 -07:00
Timo Tijhof
e8275758fe objectcache: Introduce IExpiringStore for convenient TTL constants
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.

Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
2015-10-28 04:07:25 +00:00
Ori Livneh
e9d64a69bc Replace local RunningStat with wikimedia/running-stat library
* mediawiki/vendor change: I66b419c8168df.
* Add back-compat shim in includes/compat/RunningStatCompat.php.

Change-Id: Ibad4dc0ce5aa8697791865ed8ea61a25bae7b3e0
2015-10-27 19:16:58 +00:00
Perside Rosalie
6ba942258a Changed paths to package dependency file
Paths to ./composer.lock is changed to ./vendor/composer/installed.json
and dependency information now is read from the installed.json file.
A new ComposerInstalled class has been declared and uses the class now
to access the getInstalledDependencies() method to read data

Bug: T106247
Change-Id: Ic216577bb19b4fc5832ba003fcbbe9195d707b41
2015-10-26 15:58:03 +00:00
Aaron Schulz
f36b73e96c Move some memcached classes to /libs
Also removed a few MW dependencies from MemcachedPeclBagOStuff.
It still uses an IP class method, so it has to stay for now.

Change-Id: I8c5c83046c58fb58091d6ce11b2385208262460f
2015-10-24 15:27:42 -07:00
Aaron Schulz
cce813a922 Move MultiWriteBagOStuff to /libs
Also moved related tests files to /libs.

Change-Id: I806eeaa30205733d497adde933baf0c4157f7aae
2015-10-24 12:15:26 -07:00
Aaron Schulz
660a775931 Remove old ProfilerOutputUdp class
This uses a non-standard output and requires a custom
collector that wmf does not maintain nor use anymore.

Change-Id: I41a68f7061465417fbdc5ca41f8eb6e1f99f1111
2015-10-21 20:59:47 -07:00
Reedy
8c8383a53d Remove unused ApiQueryORM
Bug: T114538
Change-Id: I4673977cef336ab3b4e375a3deab4d4fc67830eb
2015-10-15 17:01:38 +00:00
jenkins-bot
5b1d551953 Merge "Improve error message for missing content model on import" 2015-10-12 13:55:12 +00:00
This, that and the other
cdc36570ea Improve error message for missing content model on import
I was initially going to refactor out the error message into Import.php,
but it quickly became apparent that WikiImporter's error handling needs a
LOT of work. In particular, to localise the error message into the user's
language is sadly non-trivial.

Although not used currently, the MWUnknownContentModelException subclass
will help with error handling improvements in the future.

Bug: T49270
Change-Id: I9f53c9d6a8a2ea842cb2ba94d4131e10a8b08f5d
2015-10-10 19:51:20 +11:00
Aaron Schulz
a8ec2f8daf Move SquidPurgeClient under /clientpool
Each class has its own file now too.

Change-Id: I11593d6efbfce8e3981895e84edb4e0dea3998e4
2015-10-08 00:37:49 -07:00
jenkins-bot
f0b51b4200 Merge "API: Add ApiQueryAllRevisions" 2015-10-07 16:38:35 +00:00
Timo Tijhof
6d412cf028 Move WebRequestUpload class to its own file
Moving out of WebRequest.php. It's not even a subclass.

Change-Id: I87e5c01afe215a084e39b442c65d22d9d0cc1015
2015-10-06 19:24:58 -07:00
jenkins-bot
12ce3c8e8d Merge "Enforce lagged-slave read-only mode on the DB layer" 2015-10-07 01:40:34 +00:00
Ori Livneh
da3443bfa2 Add MemoizedCallable for APC-backed function memoization
Add a simple class to `libs/` for memoizing functions by caching return values
in APC. I decided not to make this an external library just yet because I see
this as potentially a part of a larger functional programming library.

Doesn't use APCBagOStuff for two reasons: (1) avoid dependency on MediaWiki
code; (2) ability to pass third &$success parameter to apc_store, to
distinguish between cache misses and cached false values.

Use this in ResourceLoaderFileModule to cache CSSMin::remap.

Change-Id: I00a47983a2583655d4631ecc9c6ba17597e36b5f
2015-10-06 23:35:03 +00:00
Brad Jorsch
2516ca03ed API: Add ApiQueryAllRevisions
Bug: T113885
Change-Id: I43bdc1e33945dab27466fc047d78af5e65df1740
2015-10-06 12:32:12 -04:00
jenkins-bot
c5ac11e2eb Merge "Start rename of DatabaseBase => Database" 2015-10-05 23:45:36 +00:00
Aaron Schulz
a88df43d3f Database debug log cleanup (remove wgDebugDumpSqlLength/wgDebugDBTransactions)
* Simplify the debug log call and use queries group
* Remove $wgDebugDumpSqlLength, as profiler output
  already has shortened query strings (one can use
  profiling without DBO_DEBUG)
* Removed $wgDebugDBTransactions as BEGIN/COMMIT already show
* Removed PostgresTransactionState as it was only used for
  $wgDebugDBTransactions handling
* This cuts down on lots of global variable usage

Change-Id: I185adb1694441d074dea965960429b4910727620
2015-10-05 12:47:09 -07:00
Aaron Schulz
db170ebe78 Enforce lagged-slave read-only mode on the DB layer
* Most callers gracefully check wfReadOnly(),
  but fail in case they dont. This also catches
  foreign DBs which might slip through the cracks.
* Also remove useless wfDebug() call around
  mDoneWrites check as write queries show in
  the logs anyway.

Change-Id: I560ebd19c4eb2b3a040d4331702346440617cfaa
2015-10-05 05:41:19 +00:00
Aaron Schulz
1b67a929ab Start rename of DatabaseBase => Database
This gives static method callers the option
to use methods like Database::factory() instead
of having to use the uglier DatabaseBase::factory().

Change-Id: I61800626b71ad2803a897df060059dbaf8778679
2015-10-04 12:36:56 -07:00
Brad Jorsch
77d62edef3 Refactor hashing utility functions from MWCryptRand and make public
MWCryptRand already has some useful utility functions wrapping PHP's
hash() and hash_hmac(). Let's make them public so we can use them from
other code.

But since "MWCryptRand" isn't really a good place for hashing functions,
let's move them to "MWCryptHash" instead.

Change-Id: I7542c719ac72beba7b0f6aa170bdb4c69fa6beab
2015-10-01 01:18:23 +00:00
jenkins-bot
2cb893c085 Merge "Introduce CategoryMembershipChange" 2015-09-30 12:44:37 +00:00
Aaron Schulz
c0cb80beac Added DeleteLinksJob to support purging backlinks via job runners
* This jobs should only be constructed via relevant Content object,
  e.g. the result of enqueueUpdate() being called on a DataUpdate
  returned by Content::getSecondaryUpdates().
* Also modified LinksDeletionUpdate to support a $pageId parameter.
* LinksDeletionUpdate can now be enqueued to a DeleteLinksJob.

Change-Id: I650dcf0bd172ede0d61357ec158a4704ae1f2033
2015-09-29 17:40:19 -07:00
addshore
2fb2a3f14b Introduce CategoryMembershipChange
This is split from:
I03516bb34144d95e5f25c46ae98ab70ce699b31b

Change-Id: I27539d25ef3e81cf991657dffc0a62b9719d21d8
2015-09-29 23:59:34 +01:00
Kunal Mehta
9a3c7b43ea Move FileContentsHasher into includes/utils/
The class only contains two dependencies upon MediaWiki (ObjectCache &
wfGlobalCacheKey) which are suitable for inclusion in the utils
directory.

Change-Id: I85b4c763be2670c40f26d93e75cedcb68eaa7987
2015-09-24 18:27:28 -07:00
jenkins-bot
827e10dc40 Merge "Use wikimedia/cldr-plural-rule-parser" 2015-09-24 23:15:56 +00:00
Aaron Schulz
9f01cee8fa Moved all load balancer classes to /loadbalancer
* Also split out LBFactory classes to their own files

Change-Id: I9e649878afcffdb736c09c0574e5f99b1e72382f
2015-09-24 22:56:05 +00:00
Niklas Laxström
4a3fd2e42a Use wikimedia/cldr-plural-rule-parser
Replaces the parser included in MediaWiki with same code in
a library.

Change-Id: I1d2675466a543269e17faf213aa68d2b7afaf78e
2015-09-24 21:41:50 +02:00
jenkins-bot
173493f0ce Merge "resourceloader: Improve caching for LESS file compilation" 2015-09-24 00:44:39 +00:00
Ori Livneh
12afb3607d resourceloader: Improve caching for LESS file compilation
Caching the output of a LESS compiler is tricky, because a LESS file may
include additional LESS files via @imports, in which case the cache needs
to vary as the contents of those files vary (and not just the contents of
the primary LESS file).

To solve this, we first introduce a utility class, FileContentsHasher. This
class is essentially a smart version of md5_file() -- given one or more file
names, it computes a hash digest of their contents. It tries to avoid
re-reading files by caching the hash digest in APC and re-using it as long as
the files' mtimes have not changed. This is the same approach I used in
I5ceb8537c.

Next, we use this class in ResourceLoaderFileModule in the following way:
whenever we compile a LESS file, we cache the result as an associative array
with the following keys:

* `files` : the list of files whose contents influenced the compiled CSS.
* `hash`  : a hash digest of the combined contents of those files.
* `css`   : the CSS output of the compiler itself.

Before using a cached value, we verify that it is still current by asking
FileContentHasher for a hash of the combined contents of all referenced files,
and we compare that against the value of the `hash` key of the cached entry.

Bug: T112035
Change-Id: I1ff61153ddb95ed17e543bd4af7dd13fa3352861
2015-09-24 00:34:17 +00:00
Aaron Schulz
25a44aa3e4 Added support for enqueueable DataUpdates
* Updates can now declare themselves as having enqueueUpdate()
  as an alternative to doUpdate(). This lets more expensive
  or slave lag producing updates use the job queue if desired.
* Added a $mode flag to DataUpdate::runUpdates() to prefer
  pushing jobs over calling doUpdate().
* Made page deletions defer deletion updates when possible.

Bug: T95501
Change-Id: Ic6f50f92768089ba0fbc223b8d178f5a91512959
2015-09-23 12:07:41 -07:00
Aaron Schulz
918fed0ab5 Moved LinksDeletionUpdate to a separate file
* Also removed unused updateCategoryCounts() method

Change-Id: I25f64d3771a23d18e7993433dab449adec375d9b
2015-09-22 11:07:47 -07:00
jenkins-bot
35f61c7c4b Merge "registration: Allow extensions to specify which MW core versions they require" 2015-09-22 04:57:47 +00:00
Erik Bernhardson
f66559b616 Produce monolog messages through kafka+avro
This allows a logging channel to be configured to write
directly to kafka. Logs can be serialized either to json
blobs or the more compact apache avro format.

The Kafka handler for monolog needs a list of one of more
kafka servers to query cluster metadata from. This should be
able to use any monolog formatter, although some like
JsonFormatter require you to disable formatBatch as Kafka
protocol would prefer to encode each record independently in
the protocol.  This requires the nmred/kafka-php library,
version >= 1.3.0.

Adds a new formatter which serializes to the apache avro
format. This is a compact binary format which uses pre-
defined schemas. This initial implementation is very simple
and takes the plain schemas as a constructor argument.

Adds a new option to MonologSpi to wrap handlers in a
BufferHandler. This doesn't flush until the request shuts
down and prevents any network requests in the logger from
adding latency to web requests.

Related mediawiki/vendor update: Ibfe4bd2036ae8e998e2973f07bd9a6f057691578

The necessary config is something like:

array(
    'loggers' => array(
        'CirrusSearchRequests' => array(
            'handlers' => array( 'kafka' ),
        ),
    ),
    'handlers' => array(
        'kafka' => array(
            'factory' => '\\MediaWiki\\Logger\\Monolog\\KafkaHandler::factory',
            'args' => array( 'localhost:9092' ),
            'formatter' => 'avro',
            'buffer' => true,
        ),
    ),
    'formatters' => array(
        'avro' => array(
            'class' => '\\MediaWiki\\Logger\\Monolog\\AvroFormatter',
            'args' => array(
                array(
                    'CirrusSearchRequests' => array(
                        'type' => 'record',
                        'name' => 'CirrusSearchRequests'
                        'fields' => array( ... )
                    ),
                ),
            ),
        ),
    ),
)

Bug: T106256
Change-Id: I6ee744b3e5306af0bed70811b558a543eed22840
2015-09-21 12:45:23 -07:00
Kunal Mehta
cef1f31167 registration: Allow extensions to specify which MW core versions they require
This adds a "requires" property to extension.json, which extensions and
skins can use to indicate which versions of MediaWiki core they support.
The hacky wfUseMW() is now deprecated in favor of this.

Rather than writing our own version constraint and parser library, we
can re-use composer's, which was recently split out into a separate
library named "composer/semver" for this patch.

Any syntax accepted by composer[1] is available for usage here. Test
cases have been provided to demonstrate how versions are parsed. For now
it is recommended that people stick to expressing compatability with
stable versions (e.g. ">= 1.26").

This patch does not support requiring specific MediaWiki core WMF
branches, since those do not follow the standard semver format that
composer parses. If we are unable to parse $wgVersion, all checking will
be skipped and reported as compatible.

[1] https://getcomposer.org/doc/01-basic-usage.md#package-versions

Bug: T99084
Change-Id: I7785827216e16c596356d0ae42d6b30f3f179f10
2015-09-21 09:56:53 -07:00
jenkins-bot
f8659ae6ea Merge "SpecialMovepage: Convert form to use OOUI controls" 2015-09-15 22:40:01 +00:00