Commit graph

625 commits

Author SHA1 Message Date
Kunal Mehta
85585c47ec Return a typed object from WikiPage::prepareContentForEdit
This makes it easier to figure out what values are available and
includes documentation about each field.

This will also allow us to add deprecation warnings for deprecated
properties via __get() in the future.

Change-Id: I4ecc900372546013253256749563aaa203ff8963
2017-07-19 06:24:39 +00:00
jenkins-bot
10d1b7d12b Merge "New maintenance script to recalculate category membership counts" 2017-07-15 08:02:21 +00:00
This, that and the other
50affc7781 New maintenance script to recalculate category membership counts
The category membership counts stored in the category table have a habit of
drifting away from the actual number of pages in the category. This script
identifies incorrectly-counted categories by performing a SELECT on the
replica DB, then querying master for up-to-date counts and updating the
category rows with those counts.

It's possible that a LinksUpdate job will be in progress while this script
is run. LinksUpdate does categorylinks INSERTs and category UPDATEs in
separate transactions, and our count might occur in between, leading to
category values that are slightly off. But since we are only updating
categories with already incorrect counts, we are not making the situation
any worse than it already was.

The script runs reasonably quickly on all but the very largest wikis.

Bug: T18765
Change-Id: I555f068fbf3aee20a7775d5532781f421f706346
2017-07-15 07:37:11 +00:00
jenkins-bot
29115b9e53 Merge "Maintenance script to purge specific page" 2017-07-13 02:18:22 +00:00
Timo Tijhof
01938ae7db statsd: Rename MediawikiStatsdDataFactory to IBufferingStatsdDataFactory
Follows-up I2874175647e (7fdc3d09a3).

* Avoid "MediaWiki" in class names. If unavoidable, always camelcase.
* Use a more descriptive name, matching the implementing the class ("Buffering").
* Use "I" prefix for consistency with other MediaWiki-provided interfaces.
  (Avoid "Interface" suffix, which is foreign to MediaWiki and is only used
  in the StatsdClient namespace, which this interface is not in).

Bug: T166354
Change-Id: I06de59122625f9c23e7c1a1bfa69a7ddabbf379e
2017-07-07 17:22:50 -07:00
Gilles Dubuc
30d645574f Maintenance script to purge specific page
Bug: T169195
Change-Id: Ia55f926e019eed5d5f421d49506b8182e5b2c5b7
2017-07-06 17:34:49 +02:00
Timo Tijhof
0df8f6d49c benchmarks: Add benchmark for JSMinPlus
Example usage

> mwscript maintenance/benchmarks/benchmarkJSMinPlus.php --file resources/lib/jquery/jquery.js -v

Bug: T31784
Change-Id: I3c11574adb864a4bfbfceba137e285fdb8dc66d2
2017-06-27 22:17:39 +00:00
jenkins-bot
57428a7c8c Merge "New language variant 'en-x-piglatin' for easier variant testing" 2017-06-14 23:57:13 +00:00
jenkins-bot
4d25a92de0 Merge "Introduce MessageLocalizer interface for exposing msg() method" 2017-06-14 18:01:51 +00:00
Liangent
d8375bee24 New language variant 'en-x-piglatin' for easier variant testing
Guarded by the $wgUsePigLatinVariant variable, off by default.

Pig Latin is a language game where words in English are altered
according to the following rules:

* Words starting with a vowel have a '-way' suffix appended.
* Words starting with a consonant have the initial consonants (or 'qu'
  group) moved to the end and an '-ay' suffix appended.

https://en.wikipedia.org/wiki/Pig_Latin

* Added 'en-x-piglatin' as a language name.
* Added 'en' to LanguageConverter::$languagesWithVariants.
* Added LanguageEn class and its corresponding EnConverter which
  provides one-way translation from English to Pig Latin.
* Some minor internal changes in code that assumed that English
  doesn't have a language class or converter.

Bug: T45547
Depends-On: I1d9691c784032669979f8109c9a5f65cbf4122c9
Change-Id: I7fa2d85d6364958c5138366e8b4504a2697a8731
2017-06-12 16:59:57 -04:00
Amir Sarabadani
bc74afc67e Move HttpAccept* to libs
Bug: T163923
Change-Id: I4984f6b77843669950afeff82351827cd416a80d
2017-06-12 18:28:25 +04:30
jenkins-bot
8b706516e0 Merge "Start a very basic version of Special:PageData" 2017-06-12 12:51:28 +00:00
jenkins-bot
137c5993e1 Merge "Move HttpAcceptNegotiator and HttpAcceptParser from Wikibase to core" 2017-06-12 12:43:59 +00:00
Amir Sarabadani
e93155cf60 Introduce MessageLocalizer interface for exposing msg() method
Bug: T162594
Change-Id: I2d9af9fc05264bffda675af46f656038cd72d0db
2017-06-12 13:34:32 +04:30
Amir Sarabadani
d5bd85c5d1 Start a very basic version of Special:PageData
Bug: T163923
Change-Id: I2a1a12f20a38d8d3c50a8f6c9a363be1cb656c70
2017-06-10 04:29:37 +04:30
Amir Sarabadani
4ec5164b4e Move HttpAcceptNegotiator and HttpAcceptParser from Wikibase to core
This will be needed for implementing Special:PageData

Bug: T163923
Change-Id: I2315d7dcdfa5973998917af311ebecc855b37f73
2017-06-10 04:27:10 +04:30
Bartosz Dziewoński
c8ad83310f Refactor registration of OOjs UI resource modules
includes/resourceloader/ResourceLoaderOOUIModule.php
* New trait centralizing some logic for dealing with OOjs UI themes,
  previously duplicated in OutputPage, ResourcesOOUI.php and
  ResourceLoaderOOUIImageModule.
* Follow-up change I74362f0fc215b26f1f104ce7bdbbac1e106736ad uses this
  as a base to allow skins/extensions to define new OOjs UI themes.

resources/Resources.php
resources/ResourcesOOUI.php
includes/resourceloader/ResourceLoader.php
* OOjs UI resource module definitions are moved back to their rightly
  place in Resources.php. They are again (almost) normal and static.
* Theme-specific logic is now handled by the module code, definitions
  only specify 'themeScripts'/'themeStyles'/'themeImages'.
* ResourcesOOUI.php is deleted and no longer loaded by ResourceLoader.

includes/resourceloader/ResourceLoaderOOUIFileModule.php
includes/resourceloader/ResourceLoaderOOUIImageModule.php
* Glue code previously existing in ResourcesOOUI.php now lives here.
* Use the ResourceLoaderOOUIModule trait to avoid code duplication.

Change-Id: I39cc2a735d9625c87bf4ede6f5fb0ec441d47dcc
2017-06-01 17:04:05 +02:00
jenkins-bot
163381a091 Merge "Refactor Statsd classes to enable null collector to work." 2017-05-30 03:41:14 +00:00
Stanislav Malyshev
7fdc3d09a3 Refactor Statsd classes to enable null collector to work.
The following changes are added:
- Created MediawikiStatsdDataFactory interface
- Added hasData() method to see if there are any data to send
- Added getData() method to fetch data
- Made service infrastructure use MediawikiStatsdDataFactory interface
- Made wfLogProfilingData() use MediawikiStatsdDataFactory interface
- Added capability to enable/disable buffering collector

Bug: T166354
Change-Id: I2874175647e987996a9a399829b3319674471aaa
2017-05-29 15:33:02 -07:00
jenkins-bot
b77e101acb Merge "Move some more classes to comply with class per file" 2017-05-25 16:15:31 +00:00
Timo Tijhof
99c1f00377 rdbms: Add compat alias for renamed LBFactory class
This was renamed to be namespaced, but there are still some uses
that justify having an alias, similar to how we did with most
other classes in rdbms that were renamed in 1.29.

Bug: T164896
Change-Id: I36426144491b976322c3c96c08a821300d3b4a46
2017-05-24 11:29:54 +00:00
Reedy
e86a6310f7 Move some more classes to comply with class per file
Change-Id: Ibecea848fcca24709847931f5e02a7c981c4d277
2017-05-21 13:23:31 +01:00
Geoffrey Mon
53fa809a62 Pager class for filtering by date range
New abstract class RangeChronologicalPager to provide shared date range
filtering capability (with unit tests)

I18n msgs to use as common labels for date range inputs

Expose some ReverseChronologicalParser::getDateCond logic so we can
convert year/month date filters to datestamp date filters

Bug: T120733
Change-Id: I65fdc00368f406f5fa2492600e95e07ce442c165
2017-05-12 14:55:54 +02:00
Brian Wolff
73f5937047 Add collation for Bashkir (ba)
This is based on a numeric uppercase collation. Bashkir characters
will be remapped to the private use area for the purpose of sorting.

Bug: T162823
Change-Id: I65f1af0b57ff6ded7d464e39efd401f178a3519e
2017-05-10 04:17:46 +00:00
Timo Tijhof
0ec9359653 resourceloader: Add CSSMin benchmarks
Usage:

 # Run default benchmark
 $ php maintenance/benchmarks/benchmarkCSSMin.php

 # Use custom file
 $ php maintenance/benchmarks/benchmarkCSSMin.php --file resources/lib/qunitjs/qunit.css

 # Debug the output for inspection (no benchmark)
 $ php maintenance/benchmarks/benchmarkCSSMin.php --out

Change-Id: I70d118131d0ff16d1a811b2de1328ea622b7ca69
2017-05-04 04:13:42 +00:00
Matthew Flaschen
b510edbda1 Revert "Convert mwdoc-filter.php to Maintenance", rm wfShellWikiCmd
Revert a58948d64 and instead remove wfShellWikiCmd and escape
shell arguments directly.

This should be fine since mwdoc-filter.php does not depend on per-wiki
state.

Change-Id: Id9c6ca84bab827675b71ca16bf688fd3f5c993a1
2017-05-04 00:15:22 +00:00
Matthew Flaschen
a58948d645 Convert mwdoc-filter.php to Maintenance class so --wiki works
Change-Id: Ie2fc4109bc2e7a23a6549d9705be44cf2eb858b9
2017-05-03 03:35:36 +00:00
Tim Starling
a59ed5f3de Add ConsoleLogger, use it for eval.php -d
eval.php previously set $wgDebugLogFile to /dev/stdout. This had the
following problems:

* It doesn't work if the maintenance script is executed via sudo, since
  /dev/stdout is typically owned by the original user, so MW can't open
  it. Using php://stdout worked on HHVM but not PHP.
* Setting $wgDebugLogFile has no effect if the wiki uses MonologSpi.
* Setting $wgDebugLogFile has no effect on channels configured with
  $wgDebugLogGroups.
* stderr is a more appropriate place to send logging output.
* Writing to configuration variables is discouraged.

So, add ConsoleSpi, which is a very simple logging service provider
which sends all messages to stderr. This should be suitable for
debugging with eval.php or shell.php in WMF production or beta.

Change-Id: Ib0d6ce45e0cbecd58263fc4e360c63d4149acb3a
2017-04-28 10:05:05 +10:00
jenkins-bot
eb72adcb4e Merge "New maintenance script to clean up rows with invalid DB keys" 2017-04-24 15:19:30 +00:00
Prateek Saxena
a6b31205ac mw.widgets: Add SelectWithInputWidget and its PHP implementation
Bug: T106999
Change-Id: Ic158bec3c463fba5099b05f41c9686f833e1c313
2017-04-24 10:19:55 +02:00
Tim Starling
448be2ed3e Add benchmarkTidy.php, to benchmark tidy drivers
Plus representative input file

Change-Id: I254793fc55c57a98c07ae1e4c27e6005965c9a20
2017-04-21 01:02:22 +00:00
Tim Starling
820f46964f A service for read-only mode
Introduce a service to represent wfReadOnly() and friends.

It's necessary to have two service instances, one for wfReadOnly() and
one for wfConfiguredReadOnlyReason(), to avoid a circular dependency,
since LoadBalancer needs the configured reason during construction, but
wfReadOnly() needs to query the currently active load balancer.

Not having a cache of the configuration makes it possible to dynamically
change the configuration. Ideally things would not change the
configuration, and I removed such instances in core, but to support
extensions, I added a test ensuring that the configuration can be changed.

Change-Id: I9bbee946c10742526d3423208efd68cb3cc5a7ee
2017-04-19 12:27:06 -07:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
jenkins-bot
0697e41485 Merge "Move Database and subclasses to Rdbms namespace" 2017-04-12 18:07:43 +00:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
jenkins-bot
bf9c2b0cb3 Merge "First version of AutoblockList special page" 2017-04-11 18:05:36 +00:00
mainframe98
680de1a04a First version of AutoblockList special page
This patch introduces a new special page named AutoblockList.
Its design is reused from Special:BlockList.

Bug: T146414
Change-Id: I811d23c98be749d8df36700b07a295355691af77
2017-04-11 10:40:42 -07:00
Aaron Schulz
9b459d29e0 Add EtcdConfig class
Bug: T156924
Change-Id: I60914d31c21484bfb935fe3d8c3168b51a2d5d1b
2017-04-10 07:36:33 +00:00
Brian Wolff
14beae88b5 SECURITY: Do not directly redirect to interwikis, but use splash page
Directly redirecting based on a url paramter might potentially
be used in a phishing attack to confuse users.

Bug: T109140
Bug: T122209
Change-Id: I6c604439320fa876719933cc7f3a3ff04fb1a6ad
2017-04-06 13:42:38 -07:00
jenkins-bot
eca93b7c40 Merge "Add class for service discovery using DNS SRV records" 2017-04-04 22:26:39 +00:00
Kunal Mehta
c186794722 Add class for service discovery using DNS SRV records
The SrvDiscoverer class can be used to find services (e.g. etcd) using
DNS SRV records.

Change-Id: Ia636d02535a3bb592eb896137cfb787a9ce6442a
2017-03-29 12:59:51 -07:00
Aaron Schulz
3c2a172c3f Move DBConnRef/MaintainableDBConnRef to Rdbms namespace
Change-Id: I7ca6b40f973b2aae83747de92e280ded222c87e3
2017-03-29 11:04:47 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Matthew Flaschen
aa063f4c5a Back-end of new RecentChanges page, refactoring
Generate old RC, Related changes (it was already displayed and working
on 'Related changes' before this change), and Watchlist/etc. and data
for new UI from back-end.

This moves everything used for defining the old (unstructured) and new
(structured) filters into unified objects, ChangesListFilter and
ChangesListFilterGroup (and sub-classes).

This includes the query logic (see below) and logic for adding
CSS attribution classes.

This is a breaking change (for subclasses of ChangesListSpecialpage)
due to the signature (and name) change of buildMainQueryConds and
doMainQuery.  An alternative that I don't think is a breaking change
would be to put the filter->DB logic in runMainQueryHook, but then it's
doing more than just running a hook.

This is because it used to only build $conds here, but it's clear from
filterOnUserExperienceLevel filters need more than this.  I added all
the DB parameters from the hook, but this could be debated.

I have an checked and fixed the WMF-deployed extensions affected by
this.

Other than that, there should be full back-compat (including legacy
filters not using the new system).

* add hidepatrolled/hideunpatrolled to new UI.

* Move userExpLevel from RC to ChangesListSpecialPage.  Although for
now the structured UI only displays on RC anyway, when it displays on
watchlist, it seems we'll want userExpLevel there.

  Change this to make 'all' exclude unregistered users.

* Don't have front-end convert none-selected to 'all' on string_options.

* Needs the hideanons/hideliu special redirect to be done before this
is merged (T151873)

Bug: T152754
Bug: T152797
Change-Id: Iec2d82f6a830403d1c948a280efa58992e0cdee7
2017-03-11 01:42:01 +00:00
This, that and the other
6519c42d24 New maintenance script to clean up rows with invalid DB keys
The TitleValue constructor, used by the link cache among other things,
throws an exception for DB keys which do not satisfy a simple sanity test
(starting or ending with _, or containing a space, tab, CR or LF
character). This has broken certain special pages on a number of WMF sites;
see T99736, T146778 and T155091.

The new cleanupInvalidDbKeys.php script allows these bogus entries to be
removed from the DB, making sure these exceptions won't be thrown in the
future. It cleans the title columns of the page, archive, redirect,
logging, category, protected_titles, recentchanges, watchlist, pagelinks,
templatelinks, and categorylinks tables.

The script doesn't support batching; most wikis should have fewer than 500
broken entries in each table. If need be, the script can be run several
times.

To make the LIKE queries work properly I had to fix the broken escaping
behaviour of Database::buildLike() -- previously it had a habit of double-
escaping things. Now an ESCAPE clause is added to change the escape
character from the problematic default backslash, and tests are added to
cover the changes.

Bug: T155091
Change-Id: I908e795e884e35be91852c0eaf056d6acfda31d8
2017-03-11 00:27:27 +11:00
jenkins-bot
bdfa96eb72 Merge "Break up $wgDummyLanguageCodes" 2017-03-08 20:46:47 +00:00
This, that and the other
48ab87d0a3 Break up $wgDummyLanguageCodes
$wgDummyLanguageCodes is a set and mapping of different language codes:

* Renamed language codes: ['als' => 'gsw', 'bat-smg' => 'sgs',
                           'be-xold' => 'be-tarask', 'fiu-vro' => 'vro',
                           'roa-rup' => 'rup', 'zh-classical' => 'lzh',
                           'zh-min-nan' => 'nan', 'zh-yue' => 'yue'].
  The old language codes are deprecated because they are invalid but
  should be supported for compatibility reasons for a while.
* Language codes of macro languages, which get mapped to the main
  language: ['bh' => 'bho', 'no' => 'nb'].
* Language variants which get mapped to main language:
  ['simple' => 'en'].
* Internal language codes of the private-use-area which get mapped to
  itself: ['qqq' => 'qqq', 'qqx' => 'qqx']

This is a very strange conglomeration which should get differentiated,
and were split up in the following ways:

* Renamed language codes are available from
  LanguageCode::getDeprecatedCodeMapping().
* Language codes of macro languages and the variants that are mapped to
  the main language are available as $wgExtraLanguageCodes and are set
  in DefaultSettings.php.
* Internal language codes are set in $wgDummyLanguageCodes in Setup.php.

Change-Id: If73c74ee87d8235381449cab7dcd9f46b0f23590
2017-03-08 12:11:30 -08:00
jenkins-bot
82524dc4da Merge "RemexHtml tidy driver with p-wrapping" 2017-03-08 15:24:36 +00:00
Tim Starling
9341a00ed1 RemexHtml tidy driver with p-wrapping
Pull in the RemexHtml library, which is an HTML 5 library I recently
created.

RemexCompatMunger mutates the event stream, inserting <mw:p-wrap>
elements where necessary, and occasionally taking even more invasive
action such as reparenting and removing nodes maintained in Serializer's
tree.

RemexCompatFormatter produces a MediaWiki-style serialization which is
relatively compatible with existing parser tests. It also does final
empty element handling, including translating <mw:p-wrap> to <p>

Tests are imported from both Html5Depurate and Subbu's pwrap.js.

Depends-On: I864f31d9afdffdde49bfd39f07a0fb7f4df5c5d9
Change-Id: I900155b7dd199b0ae2a3b9cdb6db5136fc4f35a8
2017-03-08 16:54:13 +11:00
Florian Schmidt
2d82f0881a PageArchive: Fix typo in class file
Follow up: I02b7e4785f369c7b6574c4bce093b6de58d1c651

Change-Id: Iec60fb023ef3c57e61ee29c829f750ca1b34467e
2017-03-06 22:46:55 +01:00