Commit graph

226 commits

Author SHA1 Message Date
Aaron Schulz
01c2b0a425 Add $wgCdnReboundPurgeDelay for more consistent CDN purges
Bug: T113192
Change-Id: I89deb4f8143c1cc6154cdd05bcee1f49d3e3a75a
2015-12-10 22:52:30 +00:00
Aaron Schulz
282c5fa9f3 Rename SquidUpdate => CdnCacheUpdate
Squid is not the only possible CDN

Change-Id: Ie2a2955847c5706e630322bbbab71c9d063b378f
2015-12-09 16:31:17 -08:00
jenkins-bot
03320327a5 Merge "Add $wgMaxUserDBWriteDuration to limit user-generated transactions" 2015-12-07 21:25:24 +00:00
jenkins-bot
0c33f44de9 Merge "Remove obsolete showCacheStats.php and showCacheStats.php" 2015-12-07 20:52:28 +00:00
Timo Tijhof
ade945b97a Remove obsolete showCacheStats.php and showCacheStats.php
These scripts interact with keys that used to be set by ParserCache.php.
However this hasn't been the case for a long time now. They use wfIncrStats(),
which is configured by $wgStatsdServer.

Change-Id: Id6a62aec57801085ed684af9362a96eca0914e92
2015-12-07 19:49:29 +00:00
jenkins-bot
7d78861743 Merge "tests: Remove unused TableCleanupTest class" 2015-12-06 22:32:39 +00:00
Ori Livneh
fdca49759b Follow-up for Iba0138a6d: remove now-unused MWHookException class
Change-Id: I9cf80f2b701d6db8a6cfe884e4fd15d4e0482dda
2015-12-04 13:37:18 -08:00
Aaron Schulz
c30b049580 Add $wgMaxUserDBWriteDuration to limit user-generated transactions
This is a low level catch-all net for huge updates that still slip
through. Features that let users add/remove arbitrarily many rows
to lists of arbitrary size can easily result in high lag due to
strange usage patterns or deliberate attacks.

Also removed duplicate 'autochange-username' JSON key.

Bug: T95501
Change-Id: I58a91ca23cae528ef1954d2d78c8f0a90681983e
2015-12-04 13:08:52 -08:00
Aaron Schulz
99610822d8 Add mergeable update support DeferredUpdates
* DeferrableUpdate classes can implement MergeableUpdate.
  Duplicate updates will be merged via the merge() method.
* Make SquidUpdate support merge() so that duplicate URL
  purges are now caught accross the entire pre-send request
  execution.

Change-Id: Idffdd3e71d89e4a0f28281e65a881113caae497c
2015-12-04 19:08:51 +00:00
Aaron Schulz
34dd248e28 Add pre-send update support to DeferredUpdates
* PRESEND/POSTSEND constants can now be used in addUpdate()
  and addCallableUpdate() to control when the update runs.
  This is useful for updates that may report errors the client
  should see or to just get a head start on queued or pubsub
  based updates like CDN purges. The OutputPage::output() method
  can easily take a few 100ms.
* Removed some argument b/c code from doUpdates().
* Also moved DeferrableUpdate to a separate file.

Change-Id: I9831fe890f9f68f9ad8c4f4bba6921a8f29ba666
2015-12-04 19:08:27 +00:00
jenkins-bot
323a2909ba Merge "Extracted some code from PrefixSearch for re-usability by other search engines." 2015-12-04 16:56:39 +00:00
Timo Tijhof
1a8f927d70 tests: Remove unused TableCleanupTest class
Follows-up 77086dc2ef (r56862).

Change-Id: I7e8accc35596d134c934a03072c3c3e99fd2aa0a
2015-12-03 13:45:46 +00:00
Aaron Schulz
6dedffc2d7 Move category membership RC updates to CategoryMembershipChangeJob
* Recursive link updates no longer mention an category changes.
  It's hard to avoid either duplicate mentioning of changes or
  confusing explicit and automatic category changes.
* LinksUpdate no longer handles this logic, but rather WikiPage
  decides to spawn this update when needed in doEditUpdates().
* Fix race conditions with calculating category deltas. Do not
  rely on the link tables for the read used to determine these
  writes, as they may be out-of-date due to slave lag. Using the
  master would still not be good enough since that would assume
  FIFO and serialized job execution, which is not garaunteed.
  Use the parser output of the relevant revisions to determine
  the RC rows. If 3 users quickly edit a page's categories, the
  old way could misattribute who actually changed what.
* Make sure RC rows are inserted in an order that matches that
  of the corresponding revisions.
* Better avoid mentioning time-based (parser functions) category
  changes so they don't get attributed to the next editor.
* Also wait for slaves between RC row insertions if there where
  many category changes (it theory it could well over 10K rows).
* Using a separate job better separates concerns as LinksUpdate
  should not have to care about recent changes updates.
* Added more docs to $wgRCWatchCategoryMembership.

Bug: T95501
Change-Id: I5863e7d7483a4fd1fa633597af66a0088ace4c68
2015-12-03 11:28:05 +00:00
Brad Jorsch
d032bb52cd Add a central ID lookup service
Anything that wants to be "central" right now has to depend on
CentralAuth, and then either can't work without CentralAuth or has to
branch all over the place based on whether CentralAuth is present. Most
of the time all it really needs is a mapping from local users to central
user IDs and back or the ability to query whether the local user is
attached on some other wiki, so let's make an interface for that in
core.

See I52aa0460 for an example implementation (CentralAuth), and Ibd192e29
for an example use (OAuth).

Bug: T111302
Change-Id: I49568358ec35fdfd0b9e53e441adabded5c7b80f
2015-12-02 14:50:36 -07:00
dcausse
353ae71963 Extracted some code from PrefixSearch for re-usability by other search engines.
(This patch is needed for Ida9b9f8)

Bug: T112028
Change-Id: I35aece88333a65f6b1f55f7a87e2d14de4f5bea7
2015-12-02 17:18:08 +00:00
Reedy
5d032c0f65 Rename wfBaseConvert.php maintenance script
Change-Id: I3174ceb040e9029ad8049a3e49c119bfac4aa386
2015-11-24 23:34:21 +00:00
jenkins-bot
8732532d47 Merge "Implement HTMLComboboxField" 2015-11-24 19:20:07 +00:00
jenkins-bot
899bd46cbb Merge "Add script to fix content model of JSON pages" 2015-11-24 18:53:43 +00:00
Kunal Mehta
2986d47c90 Add script to fix content model of JSON pages
MediaWiki:Foo.json and User:Foo/bar.json pages now have a default
content model of JSON, but existing pages using those names will be set
to defaults of wikitext.

The content models of those pages are now set to "json", unless it has
invalid syntax, in which case it will be set to "wikitext".

For convenience, the script is automatically run as part of update.php.

Bug: T108663
Change-Id: I1412937ccea8e65dba58580beec79cbf2286ae01
2015-11-24 10:08:44 -08:00
jenkins-bot
aa5e579028 Merge "Move details of import logs into log params" 2015-11-24 08:52:12 +00:00
Brad Jorsch
42e257a1a2 API: Remove dbg, txt, and yaml formats
Per RFC: https://www.mediawiki.org/wiki/Requests_for_comment/Ditch_crappy_API_formats

Bug: T95715
Change-Id: I8e4e9ad3c96504cf4587cb4632504eff21b5ff75
2015-11-18 09:19:51 -05:00
Bartosz Dziewoński
77d759aab4 Implement HTMLComboboxField
It's a dropdown select with the ability to add custom options, or a
text field with input suggestions, whichever you prefer.

This is meant to be a replacement for HTMLSelectOrOtherField and
HTMLAutoCompleteSelectField.

In regular HTML mode, it uses HTML5 `<datalist>` element with no
custom JavaScript. This is supported by a wide range of browsers
(IE 10+, modern Firefox and Chrome, Opera 12+).

In OOUI mode, it uses a ComboBoxInputWidget.

Depends on: I14b40884f185fb4e5

Bug: T118119
Change-Id: I954d3d24ed4efe90be9596a1bd9586ba3aee1e23
2015-11-12 09:48:27 +01:00
umherirrender
dc78d4d12c Move details of import logs into log params
The detail information about the count of revision and the interwiki
title get moved into log params. To show the new information an
ImportLogFormatter was written, which changes the key for new log items.

This allows to show the detail information in the user language.
It also decouple the user supplied data from detail information

Change-Id: Iaa57da0fc3e20c33c94fd850cd02db96fdb32dac
2015-11-11 18:34:18 +01:00
Timo Tijhof
304eaf2868 objectcache: Reduce MediaWiki-specificness from Memcached classes
Follows-up f36b73e96c, which moved these classes to libs/objectcache.

* Fix wrong @throws in MemcachedPeclBagOStuff.
* Fix wrong @returns in MemcachedBagOStuff::getClient().
* Rename MWMemcached to MemcachedClient.
* Remove mention of 'memcached.php', which doesn't exist anymore.

Change-Id: I34dbc859be4778cea489fd2344f233f30452605c
2015-11-10 03:32:58 +00:00
aude
b0c81a3e09 Remove deprecated SiteObject and SiteArray classes
SiteObject and SiteArray were deprecated in 1.21 and
can't find any use of them anywhere else in gerrit.

Change-Id: Iff3ba8a60ac9566998cce828c4034066fdefe804
2015-11-04 13:43:32 +01:00
Ori Livneh
8340400f27 Add Timing interface
* Add `Timing`, an interface which mimics the W3C User Timing API.
  It provides a canonical way to store and retrieve markers (timestamps)
  and measures (timestamps + duration).
* As the initial use-case, use it to record 'requestShutdown'.

Change-Id: I36b29162ffcc091406df025463b0e2797e52f19a
2015-11-03 15:05:37 -08:00
jenkins-bot
41f9c91545 Merge "Add script to find orphaned LocalRepo files" 2015-11-03 10:48:31 +00:00
Reedy
d9d22e8f0b Remove ORM code from core
Bug: T114538
Change-Id: Ic4196ca9da927fc5c85b01cfff65f0636e3202ae
2015-10-29 13:06:36 +00:00
Aaron Schulz
828b598911 Add script to find orphaned LocalRepo files
Bug: T111838
Change-Id: I60ac7f3451c8240a0716933dcfff413669458206
2015-10-29 01:59:10 -07:00
Timo Tijhof
e8275758fe objectcache: Introduce IExpiringStore for convenient TTL constants
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.

Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
2015-10-28 04:07:25 +00:00
Ori Livneh
e9d64a69bc Replace local RunningStat with wikimedia/running-stat library
* mediawiki/vendor change: I66b419c8168df.
* Add back-compat shim in includes/compat/RunningStatCompat.php.

Change-Id: Ibad4dc0ce5aa8697791865ed8ea61a25bae7b3e0
2015-10-27 19:16:58 +00:00
Perside Rosalie
6ba942258a Changed paths to package dependency file
Paths to ./composer.lock is changed to ./vendor/composer/installed.json
and dependency information now is read from the installed.json file.
A new ComposerInstalled class has been declared and uses the class now
to access the getInstalledDependencies() method to read data

Bug: T106247
Change-Id: Ic216577bb19b4fc5832ba003fcbbe9195d707b41
2015-10-26 15:58:03 +00:00
Aaron Schulz
f36b73e96c Move some memcached classes to /libs
Also removed a few MW dependencies from MemcachedPeclBagOStuff.
It still uses an IP class method, so it has to stay for now.

Change-Id: I8c5c83046c58fb58091d6ce11b2385208262460f
2015-10-24 15:27:42 -07:00
Aaron Schulz
cce813a922 Move MultiWriteBagOStuff to /libs
Also moved related tests files to /libs.

Change-Id: I806eeaa30205733d497adde933baf0c4157f7aae
2015-10-24 12:15:26 -07:00
Aaron Schulz
660a775931 Remove old ProfilerOutputUdp class
This uses a non-standard output and requires a custom
collector that wmf does not maintain nor use anymore.

Change-Id: I41a68f7061465417fbdc5ca41f8eb6e1f99f1111
2015-10-21 20:59:47 -07:00
Reedy
8c8383a53d Remove unused ApiQueryORM
Bug: T114538
Change-Id: I4673977cef336ab3b4e375a3deab4d4fc67830eb
2015-10-15 17:01:38 +00:00
jenkins-bot
5b1d551953 Merge "Improve error message for missing content model on import" 2015-10-12 13:55:12 +00:00
This, that and the other
cdc36570ea Improve error message for missing content model on import
I was initially going to refactor out the error message into Import.php,
but it quickly became apparent that WikiImporter's error handling needs a
LOT of work. In particular, to localise the error message into the user's
language is sadly non-trivial.

Although not used currently, the MWUnknownContentModelException subclass
will help with error handling improvements in the future.

Bug: T49270
Change-Id: I9f53c9d6a8a2ea842cb2ba94d4131e10a8b08f5d
2015-10-10 19:51:20 +11:00
Aaron Schulz
a8ec2f8daf Move SquidPurgeClient under /clientpool
Each class has its own file now too.

Change-Id: I11593d6efbfce8e3981895e84edb4e0dea3998e4
2015-10-08 00:37:49 -07:00
jenkins-bot
f0b51b4200 Merge "API: Add ApiQueryAllRevisions" 2015-10-07 16:38:35 +00:00
Timo Tijhof
6d412cf028 Move WebRequestUpload class to its own file
Moving out of WebRequest.php. It's not even a subclass.

Change-Id: I87e5c01afe215a084e39b442c65d22d9d0cc1015
2015-10-06 19:24:58 -07:00
jenkins-bot
12ce3c8e8d Merge "Enforce lagged-slave read-only mode on the DB layer" 2015-10-07 01:40:34 +00:00
Ori Livneh
da3443bfa2 Add MemoizedCallable for APC-backed function memoization
Add a simple class to `libs/` for memoizing functions by caching return values
in APC. I decided not to make this an external library just yet because I see
this as potentially a part of a larger functional programming library.

Doesn't use APCBagOStuff for two reasons: (1) avoid dependency on MediaWiki
code; (2) ability to pass third &$success parameter to apc_store, to
distinguish between cache misses and cached false values.

Use this in ResourceLoaderFileModule to cache CSSMin::remap.

Change-Id: I00a47983a2583655d4631ecc9c6ba17597e36b5f
2015-10-06 23:35:03 +00:00
Brad Jorsch
2516ca03ed API: Add ApiQueryAllRevisions
Bug: T113885
Change-Id: I43bdc1e33945dab27466fc047d78af5e65df1740
2015-10-06 12:32:12 -04:00
jenkins-bot
c5ac11e2eb Merge "Start rename of DatabaseBase => Database" 2015-10-05 23:45:36 +00:00
Aaron Schulz
a88df43d3f Database debug log cleanup (remove wgDebugDumpSqlLength/wgDebugDBTransactions)
* Simplify the debug log call and use queries group
* Remove $wgDebugDumpSqlLength, as profiler output
  already has shortened query strings (one can use
  profiling without DBO_DEBUG)
* Removed $wgDebugDBTransactions as BEGIN/COMMIT already show
* Removed PostgresTransactionState as it was only used for
  $wgDebugDBTransactions handling
* This cuts down on lots of global variable usage

Change-Id: I185adb1694441d074dea965960429b4910727620
2015-10-05 12:47:09 -07:00
Aaron Schulz
db170ebe78 Enforce lagged-slave read-only mode on the DB layer
* Most callers gracefully check wfReadOnly(),
  but fail in case they dont. This also catches
  foreign DBs which might slip through the cracks.
* Also remove useless wfDebug() call around
  mDoneWrites check as write queries show in
  the logs anyway.

Change-Id: I560ebd19c4eb2b3a040d4331702346440617cfaa
2015-10-05 05:41:19 +00:00
Aaron Schulz
1b67a929ab Start rename of DatabaseBase => Database
This gives static method callers the option
to use methods like Database::factory() instead
of having to use the uglier DatabaseBase::factory().

Change-Id: I61800626b71ad2803a897df060059dbaf8778679
2015-10-04 12:36:56 -07:00
Brad Jorsch
77d62edef3 Refactor hashing utility functions from MWCryptRand and make public
MWCryptRand already has some useful utility functions wrapping PHP's
hash() and hash_hmac(). Let's make them public so we can use them from
other code.

But since "MWCryptRand" isn't really a good place for hashing functions,
let's move them to "MWCryptHash" instead.

Change-Id: I7542c719ac72beba7b0f6aa170bdb4c69fa6beab
2015-10-01 01:18:23 +00:00
jenkins-bot
2cb893c085 Merge "Introduce CategoryMembershipChange" 2015-09-30 12:44:37 +00:00