Commit graph

652 commits

Author SHA1 Message Date
Bartosz Dziewoński
06daf807ca Split off remaining helper classes for special pages to separate files
includes/specials/formfields/
* EditWatchlistCheckboxSeriesField (for SpecialEditWatchlist)
* UploadSourceField                (for SpecialUpload)
* Licenses                         (for SpecialUpload)

includes/specials/forms/
* EditWatchlistNormalHTMLForm      (for SpecialEditWatchlist)
* PreferencesForm                  (for SpecialPreferences)
* UploadForm                       (for SpecialUpload)

includes/specials/helpers/
* ImportReporter                   (for SpecialImport)
* License                          (for SpecialUpload)

Change-Id: I58abcbb44dbf9bf1762b4252555f7552bfa7c253
2017-09-12 23:22:58 +02:00
Max Semenik
77ce3b98a0 Replace wfShellExec() with a class
This function has gotten so unwieldy that a helper was
introduced. Instead, here's this class that makes
shelling out easier and more readable.

Example usage:
  $result = Shell::command( 'shell command' )
       ->environment( [ 'ENVIRONMENT_VARIABLE' => 'VALUE' ] )
       ->limits( [ 'time' => 300 ] )
       ->execute();

  $exitCode = $result->getExitCode();
  $output = $result->getStdout();

This is a minimal change, so lots of stuff remains
unrefactored - I'd rather limit the scope of this commit.
A future improvement could be an ability to get stderr
separately from stdout.

Caveat: execution errors (proc_open is disabled/returned error) now
throw errors instead of returning a status code. wfShellExec() still
emulates this behavior though.

Competing commit: I7dccb2b67a4173a8a89b035e444fbda9102e4d0f
<legoktm> MaxSem: so you should continue working on your patch and I'll
          probably refactor on top of it later after its merged :P

Change-Id: I8ac9858b80d7908cf7e7981d7e19d0fc9c2265c0
2017-09-08 21:49:49 -07:00
jenkins-bot
c66360d167 Merge "Move OrderedStreamingForkController class from CirrusSearch to core." 2017-09-07 18:27:49 +00:00
Stanislav Malyshev
c82d30d19c Move OrderedStreamingForkController class from CirrusSearch to core.
This class may be useful for scripts outside CirrusSearch.

Change-Id: I1ed993a3b0b64d828ae5caba6c4b540b77b2f6f5
2017-09-07 10:27:43 -07:00
MusikAnimal
d09554b6ef Add basic IP range support to Special:Contributions
This works by using the new table introduced with T156318.

The only thing that differs from normal Special:Contribs is we are
showing the IP address next to each entry. This is it how it is
displayed if you request to see newbie contributions:
https://en.wikipedia.org/wiki/Special:Contributions?contribs=newbie

For the time being, Special:DeletedContributions does not support
IP ranges. Various other irrelevant links such as Uploads and Logs
are also hidden.

Refer to P4725 for a way to automate creation of edits by random
IPs in your dev environment.

IP::isValidBlock() has been deprecated with this dependent change:
https://gerrit.wikimedia.org/r/#/c/373165/

Bug: T163562
Change-Id: Ice1bdae3d16cf365da14c6df0e8d91d2b914e064
2017-09-05 16:15:33 -04:00
jenkins-bot
b936d263ef Merge "Deprecate IDatabase::nextSequenceValue()" 2017-09-01 09:39:07 +00:00
Tim Starling
95bf0043ef EtcdConfig: allow slashes in config key names
Allowing slashes in config key names allows us to trivially support the
proposed hierarchical structure with a single EtcdConfig object, by
fetching values with the relevant prefixes on startup.

Bug: T156924
Change-Id: Ica0914e61baba9c0462481925be15d79b66dc342
2017-09-01 13:42:28 +10:00
Brad Jorsch
0a9c55bfd3 Deprecate IDatabase::nextSequenceValue()
It's often forgotten because MySQL and Sqlite don't use it, the only
users are PostgreSQL and Oracle. And when used, if inserts to multiple
tables are being done it's easy to get the ordering wrong.

This patch reimplements DatabasePostgres::insertId() in terms of PG's
lastval() function, and adds triggers to the Oracle schema to make it
work the same as the other databases.

Bug: T164900
Change-Id: Ib308190c52673a9266c8495a589ae644f9fbefce
2017-08-31 11:16:26 -04:00
Brad Jorsch
11cf01dd9a Add comment table and code to start using it
A subsequent patch will remove the old columns.

Bug: T166732
Change-Id: Ic3a434c061ed6e443ea072bc62dda09acbeeed7f
2017-08-30 15:05:00 +10:00
jenkins-bot
a8ec960e9d Merge "Produce RDF dump of all categories and subcategories in a wiki." 2017-08-28 11:01:58 +00:00
Stanislav Malyshev
d9bb673b72 Produce RDF dump of all categories and subcategories in a wiki.
Example:
http://en.wiki.local.wmftest.net:8080/wiki/Category:Ducks> a mediawiki:Category ;
        rdfs:label "Ducks" ;
        mediawiki:isInCategory <http://en.wiki.local.wmftest.net:8080/wiki/Category:Birds> .

Bug: T157676
Change-Id: I59b9603581b37af59d17dd6c38247c85aee44911
2017-08-28 00:30:35 -07:00
jenkins-bot
daff410fd5 Merge "Remove MemcachedClient compat class names" 2017-08-22 15:16:13 +00:00
jenkins-bot
33bb6888c8 Merge "Remove RunningStat compat class" 2017-08-22 15:16:09 +00:00
jenkins-bot
5918cef6fe Merge "Remove IPSet compat classes" 2017-08-22 15:07:22 +00:00
jenkins-bot
05fa5fa13a Merge "Remove Cdb compat class names" 2017-08-22 15:07:06 +00:00
Kunal Mehta
434ea4f421 Remove MemcachedClient compat class names
Deprecated since 1.27, and unused in Wikimedia Git.

Change-Id: I1d73efac2fddb771124bcd31b3d40769e751410c
2017-08-21 21:50:31 -07:00
Kunal Mehta
4c6fd2bc69 Remove RunningStat compat class
Deprecated since 1.27, unused in Wikimedia Git.

Change-Id: I0086c1cd945865b10a1e7fcc34c08642db2474af
2017-08-21 21:38:16 -07:00
Kunal Mehta
015b74691b Remove IPSet compat classes
Deprecated in 1.26, unused in Wikimedia Git.

Change-Id: I7719a204160ddd0ba71a1a5d8f7088f7d552acbd
2017-08-21 21:34:50 -07:00
Kunal Mehta
7f0486b18e Remove Cdb compat class names
Deprecated since 1.25, and unused in Wikimedia Git.

Change-Id: I05bdefe2e2bf5ffcadb9846651af7367e8e7a814
2017-08-21 21:29:00 -07:00
Brad Jorsch
918e4c394c Delete maintenance/deleteRevision.php
It hasn't been updated properly since 2006 so many fields aren't being
copied to the archive table. Tim suggests it'd be best to just delete it
and, if someone needs the ability to delete or revdel revisions from the
command line, properly abstract out the deletion code instead of
duplicating it.

Change-Id: I400b8ac30b31802e7dd9f6e4d0ec10918eba0183
2017-08-21 13:30:01 -04:00
Florian Schmidt
499f643bef Move RawMessage out of Message.php to its own file
Change-Id: Idae9617dafa3c314085eb097f78f1c8d38672f31
2017-08-14 17:33:52 +00:00
addshore
c2aa8ba9ea Split ConfiguredReadOnlyMode into own file
Change-Id: I7c12a571bb5ce76c132a738b48b2b82d763a8860
2017-08-07 09:24:19 +00:00
Max Semenik
fd6e9ef2d4 Human-readable section ID support
It adds the ability to replace the current section ID escaping
schema (.C0.DE) with a HTML5-compliant escaping schema that is
displayed as Unicode in many modern browsers.

See the linked bug for discussion of various options that were
considered before the implementation. A few remarks:
* Because Sanitizer::escapeId() is used in a bunch of places without
  escaping, I'm deprecating it without altering its behavior.
* The bug described in comments for Parser::guessLegacySectionNameFromWikiText()
  is still there in some Edge versions that display mojibake.

Bug: T152540
Change-Id: Id304010a0342efbb7ef2d56c5b8b244f2e4fb2c5
2017-08-01 20:32:20 -07:00
Timo Tijhof
3bcc039e7a benchmarks: Add benchmarks for MapCacheLRU and HashBagOStuff
Currently, HashBagOStuff leads in most benchmarks, except for
the --fill benchmark, where MapCacheLRU is faster on HHVM.
(by invoking with `PHP=hhvm mwscript maintenance/bench...`)

Run in MediaWiki-Vagrant (Debian Jessie) on MacBook Pro host.

$ benchmarkLruHash.php --count 50000 --construct
Running PHP 5.6.30-0+deb8u1:
- HashBagOStuff-construct (22% faster)
  total: 446.20ms (min: 0.01ms, median: 0.01ms, max: 2.09ms)
- MapCacheLRU-construct
  total: 575.31ms (min: 0.01ms, median: 0.01ms, max: 3.51ms)
Running PHP version 5.6.99-hhvm:
- HashBagOStuff-construct (13% faster)
  total: 124.70ms (min: 0.00ms, median: 0.00ms, max: 1.27ms)
- MapCacheLRU-construct
  total: 143.76ms (min: 0.00ms, median: 0.00ms, max: 2.01ms)

For MapCacheLRU, the main slowdown is its use of Wikimedia\Assert.
Removing that would make it faster than HashBagOStuff.

$ benchmarkLruHash.php --count 3000 --fill
Running PHP 5.6.30-0+deb8u1:
- HashBagOStuff-fill (22% faster)
  total: 10196.82ms (min: 2.91ms, median: 3.24ms, max: 9.51ms)
- MapCacheLRU-fill
  total: 13197.13ms (min: 3.92ms, median: 4.31ms, max: 9.90ms)
Running PHP version 5.6.99-hhvm:
- HashBagOStuff-fill
  total: 5700.37ms (min: 1.71ms, median: 1.85ms, max: 8.24ms)
- MapCacheLRU-fill (11% faster)
  total: 4986.90ms (min: 1.48ms, median: 1.62ms, max: 6.93ms)

Change-Id: Icd03f872dddb308f162c72674c8d2aa6092395e5
2017-07-27 21:40:42 +00:00
jenkins-bot
4427b84407 Merge "Add a maintenance script for populating pp_sortkey" 2017-07-25 23:00:58 +00:00
Petr Pchelko
7fd1f6a17b JobQueue: Create a debugging queue
As a part of creating the kafka-based JobQueue implementation,
we need to transition to the new queue smoothly and be able to
first debug the event production side, then try executing a no-op
job in parallel with each of the real jobs (to be able to check
deduplication and all the jpb management code) and only then
try switching jobs one-by-one to the new backend queue.

In order to achieve that at first we need to temporary produce
both to the old job pipeline and to the new one - thus creating
a queue 'splitter' - it delegates all the methods to the main
queue while the produce requests also go to the EventBus.

This class it temporary, so it's marked deprecated right away.
It will not be needed once the queue transition is over.

Bug: T163380
Change-Id: I483b8f00f1c593ff3af6ba6826868b553339525f
2017-07-25 18:13:38 +00:00
Kunal Mehta
85585c47ec Return a typed object from WikiPage::prepareContentForEdit
This makes it easier to figure out what values are available and
includes documentation about each field.

This will also allow us to add deprecation warnings for deprecated
properties via __get() in the future.

Change-Id: I4ecc900372546013253256749563aaa203ff8963
2017-07-19 06:24:39 +00:00
jenkins-bot
10d1b7d12b Merge "New maintenance script to recalculate category membership counts" 2017-07-15 08:02:21 +00:00
This, that and the other
50affc7781 New maintenance script to recalculate category membership counts
The category membership counts stored in the category table have a habit of
drifting away from the actual number of pages in the category. This script
identifies incorrectly-counted categories by performing a SELECT on the
replica DB, then querying master for up-to-date counts and updating the
category rows with those counts.

It's possible that a LinksUpdate job will be in progress while this script
is run. LinksUpdate does categorylinks INSERTs and category UPDATEs in
separate transactions, and our count might occur in between, leading to
category values that are slightly off. But since we are only updating
categories with already incorrect counts, we are not making the situation
any worse than it already was.

The script runs reasonably quickly on all but the very largest wikis.

Bug: T18765
Change-Id: I555f068fbf3aee20a7775d5532781f421f706346
2017-07-15 07:37:11 +00:00
mdew192837
993ce4d411 Add a maintenance script for populating pp_sortkey
The addition of a pp_sortkey field in T60032 necessitates an update
to old entries in the page_props table that don't have a pp_sortkey.
The script uses the pp_value as the sort key if it's numeric.
The script extends LoggedUpdateMaintenance so it only runs once.
Added the script to MysqlUpdater so that it automatically runs.

Bug: T66949
Change-Id: Id482dc73ec1963010324e51fe9273a72dd31a7f7
2017-07-13 22:55:26 +00:00
jenkins-bot
29115b9e53 Merge "Maintenance script to purge specific page" 2017-07-13 02:18:22 +00:00
Timo Tijhof
01938ae7db statsd: Rename MediawikiStatsdDataFactory to IBufferingStatsdDataFactory
Follows-up I2874175647e (7fdc3d09a3).

* Avoid "MediaWiki" in class names. If unavoidable, always camelcase.
* Use a more descriptive name, matching the implementing the class ("Buffering").
* Use "I" prefix for consistency with other MediaWiki-provided interfaces.
  (Avoid "Interface" suffix, which is foreign to MediaWiki and is only used
  in the StatsdClient namespace, which this interface is not in).

Bug: T166354
Change-Id: I06de59122625f9c23e7c1a1bfa69a7ddabbf379e
2017-07-07 17:22:50 -07:00
Gilles Dubuc
30d645574f Maintenance script to purge specific page
Bug: T169195
Change-Id: Ia55f926e019eed5d5f421d49506b8182e5b2c5b7
2017-07-06 17:34:49 +02:00
Timo Tijhof
0df8f6d49c benchmarks: Add benchmark for JSMinPlus
Example usage

> mwscript maintenance/benchmarks/benchmarkJSMinPlus.php --file resources/lib/jquery/jquery.js -v

Bug: T31784
Change-Id: I3c11574adb864a4bfbfceba137e285fdb8dc66d2
2017-06-27 22:17:39 +00:00
jenkins-bot
57428a7c8c Merge "New language variant 'en-x-piglatin' for easier variant testing" 2017-06-14 23:57:13 +00:00
jenkins-bot
4d25a92de0 Merge "Introduce MessageLocalizer interface for exposing msg() method" 2017-06-14 18:01:51 +00:00
Liangent
d8375bee24 New language variant 'en-x-piglatin' for easier variant testing
Guarded by the $wgUsePigLatinVariant variable, off by default.

Pig Latin is a language game where words in English are altered
according to the following rules:

* Words starting with a vowel have a '-way' suffix appended.
* Words starting with a consonant have the initial consonants (or 'qu'
  group) moved to the end and an '-ay' suffix appended.

https://en.wikipedia.org/wiki/Pig_Latin

* Added 'en-x-piglatin' as a language name.
* Added 'en' to LanguageConverter::$languagesWithVariants.
* Added LanguageEn class and its corresponding EnConverter which
  provides one-way translation from English to Pig Latin.
* Some minor internal changes in code that assumed that English
  doesn't have a language class or converter.

Bug: T45547
Depends-On: I1d9691c784032669979f8109c9a5f65cbf4122c9
Change-Id: I7fa2d85d6364958c5138366e8b4504a2697a8731
2017-06-12 16:59:57 -04:00
Amir Sarabadani
bc74afc67e Move HttpAccept* to libs
Bug: T163923
Change-Id: I4984f6b77843669950afeff82351827cd416a80d
2017-06-12 18:28:25 +04:30
jenkins-bot
8b706516e0 Merge "Start a very basic version of Special:PageData" 2017-06-12 12:51:28 +00:00
jenkins-bot
137c5993e1 Merge "Move HttpAcceptNegotiator and HttpAcceptParser from Wikibase to core" 2017-06-12 12:43:59 +00:00
Amir Sarabadani
e93155cf60 Introduce MessageLocalizer interface for exposing msg() method
Bug: T162594
Change-Id: I2d9af9fc05264bffda675af46f656038cd72d0db
2017-06-12 13:34:32 +04:30
Amir Sarabadani
d5bd85c5d1 Start a very basic version of Special:PageData
Bug: T163923
Change-Id: I2a1a12f20a38d8d3c50a8f6c9a363be1cb656c70
2017-06-10 04:29:37 +04:30
Amir Sarabadani
4ec5164b4e Move HttpAcceptNegotiator and HttpAcceptParser from Wikibase to core
This will be needed for implementing Special:PageData

Bug: T163923
Change-Id: I2315d7dcdfa5973998917af311ebecc855b37f73
2017-06-10 04:27:10 +04:30
Bartosz Dziewoński
c8ad83310f Refactor registration of OOjs UI resource modules
includes/resourceloader/ResourceLoaderOOUIModule.php
* New trait centralizing some logic for dealing with OOjs UI themes,
  previously duplicated in OutputPage, ResourcesOOUI.php and
  ResourceLoaderOOUIImageModule.
* Follow-up change I74362f0fc215b26f1f104ce7bdbbac1e106736ad uses this
  as a base to allow skins/extensions to define new OOjs UI themes.

resources/Resources.php
resources/ResourcesOOUI.php
includes/resourceloader/ResourceLoader.php
* OOjs UI resource module definitions are moved back to their rightly
  place in Resources.php. They are again (almost) normal and static.
* Theme-specific logic is now handled by the module code, definitions
  only specify 'themeScripts'/'themeStyles'/'themeImages'.
* ResourcesOOUI.php is deleted and no longer loaded by ResourceLoader.

includes/resourceloader/ResourceLoaderOOUIFileModule.php
includes/resourceloader/ResourceLoaderOOUIImageModule.php
* Glue code previously existing in ResourcesOOUI.php now lives here.
* Use the ResourceLoaderOOUIModule trait to avoid code duplication.

Change-Id: I39cc2a735d9625c87bf4ede6f5fb0ec441d47dcc
2017-06-01 17:04:05 +02:00
jenkins-bot
163381a091 Merge "Refactor Statsd classes to enable null collector to work." 2017-05-30 03:41:14 +00:00
Stanislav Malyshev
7fdc3d09a3 Refactor Statsd classes to enable null collector to work.
The following changes are added:
- Created MediawikiStatsdDataFactory interface
- Added hasData() method to see if there are any data to send
- Added getData() method to fetch data
- Made service infrastructure use MediawikiStatsdDataFactory interface
- Made wfLogProfilingData() use MediawikiStatsdDataFactory interface
- Added capability to enable/disable buffering collector

Bug: T166354
Change-Id: I2874175647e987996a9a399829b3319674471aaa
2017-05-29 15:33:02 -07:00
jenkins-bot
b77e101acb Merge "Move some more classes to comply with class per file" 2017-05-25 16:15:31 +00:00
Timo Tijhof
99c1f00377 rdbms: Add compat alias for renamed LBFactory class
This was renamed to be namespaced, but there are still some uses
that justify having an alias, similar to how we did with most
other classes in rdbms that were renamed in 1.29.

Bug: T164896
Change-Id: I36426144491b976322c3c96c08a821300d3b4a46
2017-05-24 11:29:54 +00:00
Reedy
e86a6310f7 Move some more classes to comply with class per file
Change-Id: Ibecea848fcca24709847931f5e02a7c981c4d277
2017-05-21 13:23:31 +01:00
Geoffrey Mon
53fa809a62 Pager class for filtering by date range
New abstract class RangeChronologicalPager to provide shared date range
filtering capability (with unit tests)

I18n msgs to use as common labels for date range inputs

Expose some ReverseChronologicalParser::getDateCond logic so we can
convert year/month date filters to datestamp date filters

Bug: T120733
Change-Id: I65fdc00368f406f5fa2492600e95e07ce442c165
2017-05-12 14:55:54 +02:00