Commit graph

413 commits

Author SHA1 Message Date
Roan Kattouw
a4631b92e7 Improve encoding of embedded SVGs
Unencode spaces, slashes, colons and equals signs.

Bug: T175318
Change-Id: Idebdfca8f93d5e090deba5bf5a256d7054e2d6c8
2017-09-14 11:39:57 -07:00
Umherirrender
86463f3df6 build: Updating mediawiki/mediawiki-codesniffer to 0.12.0
Change-Id: Iefaae5043fa77d5d556c31079549dab8f61bd3ef
2017-09-10 21:11:37 +02:00
MusikAnimal
a23183010e Rename IP::isValidBlock to isValidRange, deprecating the former
This is to remove confusion with the MediaWiki Block class.

All instances of isValidBlock within MediaWiki core have been updated.

Usage of this function will be more widespread with this patch:
https://gerrit.wikimedia.org/r/#/c/349457/

Change-Id: Ice1bdae3d16cf365da14c6df0e8d91d2b914e067
2017-08-29 18:26:12 +00:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Umherirrender
5544cef16b Add missing type to @param documentation
Change-Id: I6b2c9c7af9a281fe457099cc3a336a60a25e74aa
2017-08-11 20:37:35 +02:00
Umherirrender
43e8cd0579 Make phpcs pass
Reenable MediaWiki.WhiteSpace.SpaceBeforeClassBrace.NoSpaceBeforeBrace,
because the mentioned bug is fixed

Bug: T172933
Change-Id: I1593bdba2295ebed401b921f2beabed69dba7638
2017-08-11 16:22:42 +02:00
jenkins-bot
3639515010 Merge "MimeAnalyzer: Add testcases for mp3 detection" 2017-08-09 16:36:23 +00:00
jenkins-bot
225c233192 Merge "rdbms: Complete coverage for Database::selectSQLText()" 2017-07-28 21:02:54 +00:00
Timo Tijhof
da6235ce3a resourceloader: Fix @covers for CSSMin tests
Change-Id: Idb451d244fbdfbb218a56c2953298b7fe29ed3d4
2017-07-27 21:40:21 -07:00
Timo Tijhof
9491b74d00 rdbms: Complete coverage for Database::selectSQLText()
Only missing cases where 'USE INDEX' and 'IGNORE INDEX'.
The test doesn't do much since the underlying methods are no-ops
by default, but at least it ensures there are no PHP errors from
these branches.

We can later re-use some of these test cases in tests specific
to one backend.

Change-Id: Id004a2ae41efaa7a367f964013e25d98ecc591ff
2017-07-27 21:30:07 -07:00
Aaron Schulz
f4e0c720a8 rdbms: Ensure onTransactionPreCommitOrIdle() callbacks don't lead transactions
If no writes started a transaction yet, the callback would run
but not commit (by design, joining the request round). Later
writes will then pile on top of it.

The point of this method is to avoid such cases, so this edge
case has been fixed.

Change-Id: I9b44b19261d679de4aff6e44a9cfeb4f684ce02e
2017-07-26 14:28:48 -07:00
jenkins-bot
07917cd2dd Merge "rdbms: Increase coverage for Database::selectSQLText()" 2017-07-25 10:20:35 +00:00
jenkins-bot
8f75b093da Merge "rdbms: Add more @covers to DatabaseSQLTest" 2017-07-25 10:20:32 +00:00
Timo Tijhof
f357c5194c rdbms: Increase coverage for Database::selectSQLText()
* Add case for `$tables[0] == ' '`.
* Add case for `$tables == ''`.
* Add case for 'DISTINCT' option.
* Add case for 'FOR UPDATE' option.
* Add case for 'LOCK IN SHARE MODE' option.
* Add case for 'EXPLAIN' option.

Change-Id: I4a5f4754bc30d31ec35a085f39321fd358b6aa49
2017-07-24 19:49:17 -07:00
Timo Tijhof
b70a3bb821 rdbms: Add more @covers to DatabaseSQLTest
Many of the main methods here have, over the years, been split up
into several protected/private methods.

Change-Id: I1b8489b1c61c0294288442a0a0cd28c9fa77f82e
2017-07-24 19:44:06 -07:00
Brad Jorsch
edf69e62d8 MultiWriteBagOStuff: Fix async writes of mutable objects
If someone writes an object into a BagOStuff, they typically expect that
later changes to the object will not affect the value stored.
MultiWriteBagOStuff's async write handling was violating this
expectation, which is potentially causing T168040.

Bug: T168040
Change-Id: Ie897b900befdc8998614af06f9339cd07665703e
2017-07-25 02:08:00 +00:00
jenkins-bot
4a5f646a7f Merge "CSSMin: Skip #default#behaviorName when detecting local files" 2017-07-24 19:49:32 +00:00
Julien Girault
032f0ce8cb CSSMin: Skip #default#behaviorName when detecting local files
Bug: T162973
Change-Id: If76869910f308f8a91c73f287e7e74c214f02e9b
2017-07-23 01:05:39 +02:00
Timo Tijhof
7928eca292 rdbms: Complete DatabaseDomain code coverage
Add missing coverage for:
* DatabaseDomain::equals()
* DatabaseDomain::newUnspecified()
* DatabaseDomain::__toString()

Change-Id: I38863c5652ec395e7194bfb85b3485e3343a3b7e
2017-07-21 20:21:39 -07:00
Timo Tijhof
ddb7575e4e rdbms: Refactor DatabaseTest
* Move DatabaseTest and DatabaseSQLTest to libs,
  and remove MediaWikiTestCase dependency.

* Refactor DatabaseTest to be a test of the Database abstract class,
  not of whatever current DB backend is configured by LocalSettings.

  - Remove most switches/conditionals and other tests for specific
    database backends. Move those to individual test classes for
    those backends instead.
  - Some tests appear to have been integration tests for the PHP driver
    and/or the db backend itself. Moved to a new DatabaseIntegrationTest.
  - Now that only the abstract Database is invoked, the test runs a bit
    faster (no real connections/queries).

* Add missing @covers tags, and remove or fix broken ones
  (follows-up 26e52f0c49).

Change-Id: I9dc4a558e701d00e95789e7eb8e02926783b65ad
2017-07-20 18:23:37 -07:00
Timo Tijhof
b857284f71 rdbms: Move DatabaseMysqlBaseTest to libs tests
* Remove MediaWikiTestCase dependency.
* Add missing @covers tags.
* Add missing visibility/static declarations.

Change-Id: I2d81a9265016174e7b4ff810f140f7d031ce9f14
2017-07-20 17:15:25 -07:00
Umherirrender
b5cddfb27b Remove empty lines at begin of function, if, foreach, switch
Organize phpcs.xml a bit

Change-Id: Ifb767729b481b4b686e6d6444cf48b1f580cc478
2017-07-01 11:34:16 +00:00
jenkins-bot
b4ba469f26 Merge "Record TransactionProfiler violations beyond the first-by-type" 2017-06-28 01:13:14 +00:00
Aaron Schulz
e447f90896 Record TransactionProfiler violations beyond the first-by-type
Beyond giving a more complete picture, this can be used to adjust
the PostSend criteria to log master queries under the restInPeace()
entry point.

Bug: T166199
Change-Id: If5cf704ba10291d0eba63e66b388666fde3ca649
2017-06-27 16:44:33 -07:00
jenkins-bot
bc213a9d46 Merge "build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0" 2017-06-27 08:57:28 +00:00
jenkins-bot
8ddf721b6f Merge "Remove "@author Bryan Davis" and "Brad Jorsch" annotations" 2017-06-27 06:02:23 +00:00
Bryan Davis
aaf1e17088 Remove "@author Bryan Davis" and "Brad Jorsch" annotations
Undo traces of a practice we carried over from past projects and
existing examples that is neither universal nor actively encouraged in
the MediaWiki codebase.

Bug: T139301
Change-Id: I5c9c89b72a45a44aa4264a5e57b003c1a86cdf6e
Co-Authored-By: Brad Jorsch <bjorsch@wikimedia.org>
2017-06-26 15:25:46 -06:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
Timo Tijhof
dd629564df objectcache: Forward MultiWriteBagOStuff::makeKey to primary backend
Similar to what WANObjectCache and CachedBagOStuff are already doing.

Also add missing tests for WANObjectCache (similar to those for CachedBagOStuff).

Bug: T167465
Change-Id: I1a0c9324726aa6a1b221def985773b1b819181fd
2017-06-14 18:11:55 +01:00
Amir Sarabadani
bc74afc67e Move HttpAccept* to libs
Bug: T163923
Change-Id: I4984f6b77843669950afeff82351827cd416a80d
2017-06-12 18:28:25 +04:30
Derk-Jan Hartman
c6235442b4 MimeAnalyzer: Add testcases for mp3 detection
This is a follow-up to change Ie1a63aa

Bug: T115170
Change-Id: Iab5e19e1dd26b0d88c72ad7253cd11e865c5e008
2017-06-08 00:27:33 +00:00
Aaron Schulz
83fcb86dfa objectcache: add getMultiWithUnionSetCallback() method
This supports callbacks that fetch all the missing values at once.

Change-Id: I74747cc06f97edc9163178180597e6651743b048
2017-05-30 23:34:28 +00:00
Aaron Schulz
6ddaf52d32 objectcache: respect process cache in getMultiWithSetCallback()
Backend cache queries are now avoided if they are fresh in memory.

Also factor out some code into private methods for clarity.

Change-Id: Ib343fde3dbf63f39e8cb09eca6278811a7d9738b
2017-05-26 15:34:45 -07:00
Aaron Schulz
0c24cefa12 objectcache: fix cache warmup bug in getMultiWithSetCallback()
The warmup cache was not properly prefixed and was also using the entity
IDs instead of the cache keys. Thus, it effectively just wasted a
getMulti() query and resulted in the usual separate GETs anyway.

Added some unit tests for this.

Change-Id: I75b7a31214b515511856f9d95db32e8881d80ccc
2017-05-26 04:28:13 +00:00
Derk-Jan Hartman
f441e09c9a Tests: correct order of assertEquals arguments
Change-Id: Ib8d1e8aa92172a9abc0495dc0c18ea5df850dd5e
2017-05-19 13:41:16 +02:00
jenkins-bot
dca600fe7c Merge "CSSMin: Support parenthesis and quotes in url references" 2017-05-05 18:58:22 +00:00
Timo Tijhof
bb36c7b32a CSSMin: Support parenthesis and quotes in url references
Previously they were often being cut short due to the url pattern
ending at the first single quote, double quote or closing parenthesis
regardless of which of those started the url match.

Running benchmarkCSSMin.php before and after the change doesn't seem
produce consistent improvement or regression. Repeated runs with count=100
with and without this change both have a median between 2.6ms and 2.9ms
using PHP 5.6, and between 2.6ms and 2.8ms using HHVM 3.12.

Bug: T60473
Change-Id: I6d6a077ad76588f3ed81b1901a26b7e56d2157ee
2017-05-05 18:33:24 +00:00
Paladox
54c56da85a Fix php code style
Preparation change for updating mediawiki code sniffer to 0.8.0

Change-Id: Ib0b3fe4afea9096ffa3a1347b4f7e07d3398b0b2
2017-05-05 12:03:54 +00:00
Gergő Tisza
525bfbc8df Switch to librarized version of TestingAccessWrapper
Replaces \TestingAccessWrapper (defined in core) with
\Wikimedia\TestingAccessWrapper (defined in the composer package
wikimedia/testing-access-wrapper).

See https://gerrit.wikimedia.org/r/#/q/topic:librarize-testing-access-wrapper
for downstream patches.

The core version of the class is kept around for a while to avoid
circular dependency problems.

Bug: T163434
Change-Id: I52cc257e593da3d6c3b01a909e554a950225aec8
2017-04-20 14:15:57 +00:00
Timo Tijhof
0977b6c1f6 phpunit: Avoid use of wmf-production host names
* Remove used of 'eqiad' and 'wmnet' in unit tests.

Change-Id: I5bf19b63876e4dc8deaca8cd9907dca0bfa15455
2017-04-12 02:53:33 +00:00
Aaron Schulz
9b459d29e0 Add EtcdConfig class
Bug: T156924
Change-Id: I60914d31c21484bfb935fe3d8c3168b51a2d5d1b
2017-04-10 07:36:33 +00:00
Timo Tijhof
8e81621dcf objectcache: Complete code coverage for CachedBagOStuff
Change-Id: I8a228d68701f1ad4d37f60de53d105c32898dc8b
2017-04-07 23:31:41 +00:00
Timo Tijhof
a116bc58b5 objectcache: Fix CachedBagOStuff to use backend makeKey()
Follows-up 25dbd91513.

Change-Id: Ib727c57cb27f05c0462bfdfee89a185ef6603ddd
2017-04-07 23:31:25 +00:00
Timo Tijhof
447ce7e39a phpunit: Avoid use of deprecated getMock for PHPUnit 5 compat
The default will remain PHPUnit 4.x due to PHP 5.5 support.

But, we should allow developers to run tests with newer PHPUnit
versions which are noticably faster (especially for code coverage
reports).

* <https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-5.4.0>
  PHPUnit 5 deprecates the getMock() shortcut for getMockBuilder()->getMock().
  It instead introduces the shortcut createMock() which has better defaults
  than getMockBuilder(). For example, it sets 'disableArgumentCloning' and
  other things by default.

  Going forward, code should either use getMockBuilder directly and configure
  it using the setter methods (instead of the confusing variadic arguments
  of getMock) or simply use the new minimalistic createMock method. This patch
  backports the createMock method to MediaWikiTestCase so that we can start
  using it.

Change-Id: I091c0289b21d2b1c876adba89529dc3e72b99af2
2017-04-06 00:44:32 +00:00
jenkins-bot
eca93b7c40 Merge "Add class for service discovery using DNS SRV records" 2017-04-04 22:26:39 +00:00
jenkins-bot
4d11ecc013 Merge "ObjectFactory: Complete code coverage for ObjectFactoryTest" 2017-04-01 02:59:01 +00:00
Timo Tijhof
8bada02a0f ObjectFactory: Complete code coverage for ObjectFactoryTest
Cover missing case of expandClosures() where the array contains
both a closure and a non-closure.

Change-Id: I30ea8cf3fb909a499a95bf9bd24792f4dd6b5c64
2017-03-31 18:07:59 -07:00
Timo Tijhof
751f2b9d25 objectcache: Complete code coverage for HashBagOStuff
Change-Id: I06cb9778df8239706bf82c0c39cccb419eade4b2
2017-03-31 18:03:38 -07:00
Kunal Mehta
c186794722 Add class for service discovery using DNS SRV records
The SrvDiscoverer class can be used to find services (e.g. etcd) using
DNS SRV records.

Change-Id: Ia636d02535a3bb592eb896137cfb787a9ce6442a
2017-03-29 12:59:51 -07:00
Thiemo Mättig
549c1f6dd7 Add tests for CSSMin::getMimeType
Direct follow up for I1768646. The tests are copied from my patch
If985f33, with one important difference: Now the file extension always
wins. Mismatches are not fixed any more by this code. This is intended.
See the discussion at I1768646.

Change-Id: I06b873a808d58ae7e53272765b156ac0fa293c2c
2017-03-26 18:08:06 +02:00
Kunal Mehta
a57b64436c Use wikimedia/timestamp
Bug: T100924
Depends-On: I0a067367cda6885fa45631ed7c18799d653dc9bf
Change-Id: I69ba64e364df8af089c1c918cdf32f99454e693a
2017-02-28 21:15:38 -08:00
Aaron Schulz
64df456b39 Move remaining LoadBalancer classes to Rdbms
The old names are left as aliases.

Change-Id: I52a327f2463a2ba7437324047b5b00d28cd1d758
2017-02-23 20:38:31 -08:00
James D. Forrester
1e9c361960 tests: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I46261416f7603558dceb76ebe695a5cac274e417
2017-02-21 02:14:34 +00:00
Aaron Schulz
38b31bc8db Move DatabaseDomain to Rdbms namespace
Change-Id: Ifb06e792a36b5123ec3596933d0d394711ee5d08
2017-02-07 13:21:40 -08:00
Aaron Schulz
2e5eb693de objectcache: add WANObjectCacheReaper for assuring purges
* This fixes keys based on some sort of change log.
  Updates are wrapped in a mutex and keep track of the
  last known good position.
* Make WANObjectReapUpdate class that cleans up title
  related keys using the recentchanges table. This triggers
  as a deferred updates on RC view.

Change-Id: I7f14b9ca2533032147e62b1a3cc004a23da86579
2017-01-27 02:21:06 +00:00
Brion Vibber
bf2c7b02b8 Add test case for Opus file check
Test case for Opus file check so we don't regress!
Confirms the fix for T151352 that Opus files return
type 'audio' and not 'multimedia'.

Bug: T151352
Change-Id: Ib868793d90b60609ccec25726e6fbaa28bffebaf
2017-01-17 13:24:16 -08:00
addshore
05ab20fcc6 Remove Atomic methods from ConnectionManagers
Change-Id: I697f63f45975b13af52bc22fced6e4d07e35115f
Depends-On: I4341a1b4ff6a67e4c1770faae38e9b126f1bf0bf
2016-12-02 04:50:29 +00:00
addshore
a4bd573142 Fix SessionConsistentConnectionManagerTest class and file name
Change-Id: If60f081946b8f3bb564f2bb17fd1261af0cb7e3a
2016-12-01 00:15:57 +00:00
addshore
c3c3cf9696 Add DB ConnectionManagers
This moves and refactors the ConsistentReadConnectionManager
from Wikibase into the core rdbms lib.
The refactoring also creates a generic ConnectionManager.

This relates to Iff20a22f9f2bc7ceefd6defc0ed9a494a6fe62c0
which introduced a DB factory / connection manager in
an extension revealing the need for this in multiple places.

Change-Id: I0c58e15aed5bed88323d18cb95e5008f8d3381c5
2016-11-30 11:43:22 +00:00
Hashar
4d487d1942 Revert "objectcache: detect default getWithSetCallback() set options"
Causes HHVM 3.12.7 to segfault and or:
Fatal error: Stack overflow in includes/libs/objectcache/BagOStuff.php on line 754

This reverts commit b47ce21cec.

Bug: T150833
Change-Id: Ia52fdf65f5e386c0eaa1046328680fb5c8a9081a
2016-11-16 10:35:27 +00:00
Aaron Schulz
b47ce21cec objectcache: detect default getWithSetCallback() set options
This works by setting a callback to return the cache set
options. The callback will watch DB reads and create a
merged result from said usage.

This handles callers that are missing getCacheSetOptions().

Change-Id: Ia264f011e45e8cf105480955dad7e2c4c2357b73
2016-11-16 04:53:53 +00:00
Bartosz Dziewoński
c48e061f67 CSSMin: Correctly avoid fallbacks when embedding SVG files
The check for file existence before checking its MIME type was
building the path incorrectly, causing this piece of code to always
think the file does not exist. (The code to actually embed the file
was correct.)

This was not caught by the tests before of a bug in the test code,
which cancelled out this one.

Bug: T100958
Change-Id: I4469079f41d48052d0a02f98bb1ed02407abd54c
2016-11-02 10:26:21 +01:00
Aaron Schulz
43ff2a83b5 objectcache: avoid using process cache in nested callbacks
Because the process cache can be lagged by virtue of blind TTL,
the HOLDOFF_TTL might not be enough to account for it, so avoid
using it when already inside a callback.

Also split of the tests from the MediaWiki test class, so this
does not require DB access anymore.

Change-Id: I743a1233a5efc7f036fad140a9ff8a30b32f8f27
2016-10-20 22:13:29 +00:00
Kunal Mehta
75fefc066b checkComposerLockUpToDate: Always check dependencies
Upstream composer has replaced the 'hash' with a smarter 'content-hash',
but instead of re-implementing (or copy-pasting) all of that in MediaWiki
we can just compare the dependencies themselves, since that's all we
care about.

Bug: T147189
Change-Id: Ic2f22a82699e2b707b6ccb355605999a183a56a0
2016-10-16 12:49:41 -07:00
Aaron Schulz
0d87d67d61 Move MimeMagic code to libs/mime/MimeAnalyzer.php
* The later resides in /libs with related files.
* Explose MimeAnalyzer as a service.
* Keep MimeMagic::singleton() as a b/c alias.
* MimeMagic::applyDefaultConfig() will bootstrap the service
  with all of the old config, extension hook handler, and
  detector command shell-out behavior.

Change-Id: Ie2695a52e7a3bcfda9f7fa83659a9ff31b372bc3
2016-10-15 00:05:16 +00:00
Timo Tijhof
1cdecd581d objectcache: Fix test coverage
Follows-up 24200e8, 37df4f8. Test coverage still broken.

> Trying to @cover not existing method
> "WANObjectCache::getMultitWithSetCallback".

Change-Id: Ie4c9a998ea352f93f3c7dcdea25ea5c017ffbab5
2016-10-11 21:48:44 +02:00
Timo Tijhof
37df4f8648 objectcache: Fix test coverage
Follows-up 24200e88d2. This broke the test coverage build.

Change-Id: Ied93e8ce280accc842418069883ef60cd4e08441
2016-10-11 01:25:30 +02:00
Aaron Schulz
24200e88d2 objectcache: add WANObjectCache::getMultiWithSetCallback()
This does what it says on the tin, e.g. a way to fetch
multiple keys at once. Callbacks are still called in a
serial manner when needed.

Change-Id: I8e24a6de7f46499a53ec41636c5a4f106b9b3d09
2016-10-05 15:50:18 +00:00
jenkins-bot
81667f1384 Merge "Improve @covers tags for ComposerJson and ComposerLock tests" 2016-10-04 21:31:34 +00:00
Kunal Mehta
8e7803aae9 Improve @covers tags for ComposerJson and ComposerLock tests
Indicate that the constructors are also covered in the tests.

Change-Id: Ic87603c76e8affc31f497962375c39fb2726e604
2016-10-02 02:22:10 -07:00
Reedy
c214c9a952 Detect/use APCu properly
In PHP 5.5 and above, userland APC caching moved to an extension

Bug: T140587
Change-Id: Ie0871776cd7e67838471a4fe95451cf4164079b7
2016-10-01 23:07:47 +01:00
Kunal Mehta
090d0267da Use wikimedia/wait-condition-loop
Since the WaitConditionLoop class was first introduced in 1.28 (current
master), no back-compat alias is added.

Bug: T146256
Depends-On: Ia84774d83da79fea1e167fe065c69549981f753b
Change-Id: Ibd4f15c87105b8caccbd1f661b74b6efa012b77f
2016-10-01 00:25:43 -07:00
Kunal Mehta
5320f0835e Use wikimedia/scoped-callback
The ScopedCallback class was moved into a separate library. This updates
all callers to use the namespaced version, and provides a
backwards-compatibility class wrapper under the old name.

Bug: T146258
Change-Id: I2dd0a66fe2f510f26bdfef6b0a975c1beb3fd93c
Depends-On: Iea0c40bdd7776372ccf72db8f088a2abaa4d3721
2016-09-29 10:41:40 +00:00
Kunal Mehta
8e9d2e93ca Rename Convert*a*bleTimestamp to Convert*i*bleTimestamp
Per the dictionary.

Change-Id: I3dcef9fb020c8c6764ee073111ed8c032e4c5a63
2016-09-21 21:43:32 -07:00
jenkins-bot
b35ff1b4ee Merge "time: Implement ConvertableTimestamp::convert()" 2016-09-22 03:55:33 +00:00
Timo Tijhof
c46aa59dad time: Implement ConvertableTimestamp::convert()
* This method is analogous to wfTimestamp(). Optimise for the common
idiom of just converting a timestamp without having the caller hold
on to any object.

* Make wfTimestamp() use this (it could already since it didn't
use any MWTimestamp methods). Use via MWTimestamp. While this is
the same as direct access, it allows future changes.

* Add tests covering this new method.

Change-Id: I7f9104f1701d92fe25d72c7943581c64e1d093fa
2016-09-22 03:38:07 +00:00
Kunal Mehta
04dfd9b6bc Move IP class to libs/
Also fix some misplaced parenthesis in IPTest.

Change-Id: I84d6120c49f733ec45e7e0005259871808b7568b
2016-09-21 20:18:08 -07:00
Kunal Mehta
8669bced5a Move XMP* classes to includes/libs
These no longer have any dependencies upon MediaWiki and can be split
out into a separate library.

Tests were updated for the new location and to no longer depend upon
MediaWikiTestCase.

Bug: T100922
Change-Id: I6ad507fa883d5934b807f9e31c507659d0730b52
2016-09-20 20:20:10 -07:00
Aaron Schulz
847b91bf1f Make database classes handle hyphens in $wgDBname
* Add DatabaseDomain class to handle passing domains around.
It also can be cast to and from strings, which are of the same
format as wfWikiId() except with hyphens escaped.
* Make IDatabase::getDomainID() use these IDs so they can be
passed into LoadBalancer::getConnection() and friends without
breaking on sites with a hyphen in the DB name.
* Add more LBFactory unit tests for domains.

Bug: T145840
Change-Id: Icfed62b251af8cef706a899197c3ccdb730ef4d1
2016-09-17 15:29:21 -07:00
Aaron Schulz
e10f272a05 Split out ConvertableTimestamp class
* Make TimestampException a regular exception.
* Make DatabaseBase use ConvertableTimestamp.

Change-Id: I849737c9c187e8e0c6e080887ec0656e034f80d2
2016-09-16 03:00:09 +00:00
jenkins-bot
945b8ee0d4 Merge "Switch some callers to WaitConditionLoop" 2016-09-09 02:57:13 +00:00
Aaron Schulz
d32b64235f Switch some callers to WaitConditionLoop
Also fixed up backwards documentation

Change-Id: I00c36aa751a79ca86a754e049a6da78cbb417b81
2016-09-08 19:29:18 -07:00
Aaron Schulz
8db4c163e8 objectcache: pass in the $oldValue as-of time in getWithSetCallback()
This lets callers use adaptive TTLs on the near-expiration
preemptive refreshes if the new and current values match,
using the as-of time as $mtime.

Change-Id: Ie541c35f890c9f789d1accf9f2a43506daaf31f0
2016-09-08 18:36:25 +00:00
Aaron Schulz
5316e7d700 objectcache: add "pcGroup" option to WANObjectCache::getWithSetCallback()
This lets callers have their own process cache but keep it
managed in a central location, making it easier to reset
when DB transaction snapshot are flushed or when unit tests
want to clear caches between runs.

Use this to replace the Revision text process cache.

Change-Id: Ic61ee9140d4ce9836cc4650adb5bb75a291fea18
2016-09-07 16:35:15 -07:00
Aaron Schulz
3348200473 Make adaptiveTTL() less strict about $mtime type
Callers using wfTimestamp( TS_UNIX, ... ) where getting $minTTL
due to the output being a string number.

Change-Id: I6b67a941940f40ef9a543f11d0dbccacafaaa53b
2016-09-07 09:18:37 -07:00
jenkins-bot
a303296f27 Merge "Make statsd sampling rates configurable" 2016-09-05 08:52:54 +00:00
Aaron Schulz
e99bb1b7dc objectcache: allow for callbacks to mask SYNC_WRITE latency
Change-Id: I908222ad3788ebe330aa58831cda139da32becd8
2016-09-01 13:44:18 -07:00
Aaron Schulz
52511952de objectcache: add and use adaptiveTTL() method
* This better handles delayed/lost cache purges by
  having lower TTLs for entries that often changes.
* Use this for foreign upload description page caches,
  we purges are never received from the source wiki.
* Also use this for User and LocalFile cache TTLs.
* Also move the Database::getCacheSetOptions() call in
  User *before* doing the queries, which is preferred.
* Fixed some IDEA errors too, like the undeclared
  mApiBase field.

Change-Id: I70f8ebb29ac853c2a530d9eedb9e7facc1b7b710
2016-08-31 03:40:56 +00:00
Timo Tijhof
6d43bd7186 objectcache: Add missing @covers to unit tests
* HashBagOStuff: 100%
* CachedBagOStuff: 64%
* MultiWriteBagOStuff: 33%

Change-Id: I50bb8f5eda7eabadb5fd4b841af42b3bbcaf9611
2016-08-22 23:28:51 -07:00
jenkins-bot
b33cbac370 Merge "objectcache: add mcrouter support to WANObjectCache" 2016-08-19 02:57:18 +00:00
Aaron Schulz
70547f3fa3 objectcache: add mcrouter support to WANObjectCache
* Update documentation about relay methods.
* Change interim key set() to add() to avoid broadcasting it.
* Remove the behavior of doing purges synchronously in the
  local DC first before relay. In both the event relayer and
  mcrouter case, they will be asynchronous. It was hardly
  even possible to use such behavior since loads come from
  slave DBs, which do not see changes right after COMMIT.

Bug: T97562
Change-Id: I7759c82ae6e1b72fc227882a99c9a712a46374f6
2016-08-18 22:42:11 +00:00
Timo Tijhof
734f015035 ObjectFactoryTest: Add tests for 'factory' option
Increase coverage of getObjectFromSpec():

* Case for 'factory' option.
* Case for 'class' option with no arguments.
* Case for missing 'factory' and 'class' options.

Change-Id: Idc9ee73de4f6e55372b4ab5b1afbaa4c7e54509a
2016-08-17 09:32:31 -07:00
Timo Tijhof
6bb00b92bb phpunit: Add @covers to ObjectFactoryTest
* expandClosures() wasn't covered at all.
  Make the test dedicated to this feature cover that method.

* constructClassInstance() coverage was almost complete except for
  the exception case. This was covered in a separate test, but it
  was missing its @covers tag.

Change-Id: I73252849a92440fd7ae2c44f9ca58f7bf9c1f92c
2016-08-17 09:11:51 -07:00
Aaron Schulz
55f6639c2e objectcache: Add changeTTL() method
This can change the TTL without fetching the object
so that keys can be renewed or set to expire soon.

Change-Id: Id1c2c9f89b3445cfa34263057dc5029cbe170833
2016-08-11 19:55:32 -07:00
Aaron Schulz
1a081c6707 objectcache: Add "busyValue" option to WANObjectCache::getWithSetCallback
This is useful for avoiding stampedes in the one case that lockTSE
does not alone cover, which is when the key does not exist or is
tombstoned.

Also avoid saving interim values unless the key is tombstoned
since there is no point in doing that otherwise.

Change-Id: I70997e90217a0979e0589afa7a5107b0e623c7cf
2016-07-22 22:24:38 +00:00
Brian Wolff
da638d6b6e Add a test for entity expansion OOM in XmlTypeCheck
Change-Id: Ia255178fc893a715fd44e5c69399575ee7b12cfd
2016-07-06 18:56:21 -04:00
Aaron Schulz
fd4940a1b6 Fix WAN cache stashing bug due to missing list() call
Fixes regression from 0a38dbc809.

Bug: T137877
Change-Id: I4ea960966c084ada36dc54d4ec8729126c17ccdc
2016-06-15 09:40:34 -07:00
Aaron Schulz
0a38dbc809 objectcache: Support key versioning in WANObjectCache
* getWithSetCallback() takes a 'version' parameter.
* If the value at a key has a different version, then
  getWithSetCallback() will automatically use a separate
  key. Which value "wins" the main key does not matter.
* Purges are handled by using the main key as a sort of
  check key (with no hold-off). Note that this key is always
  purged on delete().
* Changed stash keys to track the same info as other keys
  both for consistency and because this change needs the
  generation timestamp. Renamed the stash prefix to avoid
  corrupt results with Het Deploy.
* This is useful for things like the User class that use
  versioning and have cross-wiki key access and purges.
  Currently, bumps to version must be deployed to all wikis
  at once, which this aims to avoid.

Change-Id: I26ae62f116e32b48bcf06bc13f8b9e79ae976745
2016-05-25 18:03:21 +00:00
Kunal Mehta
67e62c0b25 Add LinkRenderer (rewrite of Linker::link())
This is a rewrite of Linker::link() to a non-static, LinkTarget-based
interface. Users of plain Linker::link() with no options can use the
LinkRenderer instance provided by MediaWikiServices. Others that
have specific options should create and configure their own instance,
which can be used to create as many links as necessary.

The main entrypoints for making links are:
* ->makeLink( $target, $text, $attribs, $query );
* ->makeKnownLink( $target, $text, $attribs, $query );
* ->makeBrokenLink( $target, $text, $attribs, $query );

The order of the parameters are the same as Linker::link(), except
$options are now part of the LinkRenderer instance, and
known/broken status requires calling the function explicitly.
Additionally, instead of passing in raw $html for the link text, the
$text parameter will automatically be escaped unless it is specially
marked as safe HTML using the MediaWiki\Linker\HtmlArmor class.

The LinkBegin and LinkEnd hooks are now deprecated, but still function
for backwards-compatability. Clients should migrate to the nearly-
equivalent LinkRendererBegin and LinkRendererEnd hooks.
The main differences between the hooks are:
* Passing HtmlPageLinkRenderer object instead of deprecated DummyLinker
* Using LinkTarget instead of Title
* Begin hook can no longer change known/broken status of link. Use the
TitleIsAlwaysKnown hook for that.
* $options are no longer passed, they can be read (but shouldn't be
modified!) from the LinkRenderer object.

Bug: T469
Change-Id: I057cc86ae6404a080aa3c8e0e956ecbb10a897d5
2016-05-23 12:00:09 -07:00
Gergő Tisza
b203ec5fcd Make statsd sampling rates configurable
Change-Id: I6f5b0531ff5caf4b0fda38331fabad93a6a8a360
2016-05-16 14:37:30 +00:00
Ori Livneh
20abb707ba Decouple Xhprof profiling from profiling data processing
The motivation for this patch came from trying to use xhprof to profile the
unit tests. I was able to profile specific test suites, but if I tried to
profile a complete PHPUnit run, I ended up with empty profiling data. My
initial suspicion was that this was due to some Xhprof buffer getting
exhausted. The actual reason ended up being much simpler: the XhprofTest suite
indirectly called xhprof_enable() / xhprof_disable(), which stopped xhprof and
cleared out the data, so that when I was calling xhprof_disable() at the end of
the run, there was no profiling data to return, because xhprof was not running.

For the most part the XhprofTest was already doing the right thing by trying to
avoid having side-effects or relying on xhprof. Wherever possible, test fixture
profiling data was used in lieu of actually running xhprof. But this was not
totally successful because the Xhprof class coupled the collection of data to
the processing of data. Xhprof::__construct() called xhprof_enable(), so there
was no real way around that.

I think that the right way to fix that is to decouple profiling from profiling
data analysis. Thus I renamed 'Xhprof' to 'XhprofData', and modified the class
so that it expects to be fed profiling data rather than going out and
collecting it on its own. As a result, it is now possible to profile a full
phpunit run with xhprof, and the work that went into writing fixtures for the
Xhprof unit tests pays off: the class and the tests no longer have a hard
dependency on the xhprof extension, and the tests do not have to be skipped
when it is not installed. And the tests are really testing the system under
test, rather than the xhprof extension.

Finally, I added a new Xhprof class, which really is just an extremely thin
wrapper around xhprof_enable() / xhprof_disable(). The only extra functionality
it provides is the ability to check whether xhprof is running, via
Xhprof::isEnabled(). Calling Xhprof::enable() when it is already enabled will
cause an exception to be thrown. This should help us avoid running into
situations where two components contend for control of the profiler without
realizing it. A unit test tests this behavior.

The only part of this change that is not covered by tests is the change to
ProfilerXhprof. I tested it manually and it works.

Change-Id: Ica96beded68f04083abaf48ef1ae8c726eb60fa6
2016-05-12 22:23:05 +00:00
Bryan Davis
450bb6723d BagOStuff: Optionally log duplicate key lookups
Add new 'reportDupes' parameter to BagOStuff. This parameter enables scheduling
of a callback after the current web request ends. This callback will emit warning
log messages for all keys that were requested more than once.

The default ObjectCache factory for MediaWiki enables this option by default.
Not by default for plain BagOStuff instances, however. E.g. `new HashBagOStuff()`.

It also set 'asyncHandler' for all classes now (not just MultiWriteBagOStuff).

Bug: T128125
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: I8a2b06cf54d2acf5950eed71756ecdf50e224be1
2016-04-22 04:21:20 +00:00
Kevin Israel
5b48bf1b92 Clean up after "Kill mbstring fallbacks"
* Removed fallback code from Language, the associated data file
  (Utf8Case.ser), and the code to generate that data file.
* Removed comment in LanguageFi that "mb_substr has a compatibility
  function in GlobalFunctions.php".
* Removed check for mbstring in bench_utf8_title_check.php.
* In the tests for StringUtils::isUtf8():
  * Removed separate test for the non-mbstring code path.
  * Removed mentions of mbstring from function names and assertion
    messages, since mb_check_encoding() is now always used.
* Also updated the comment in StringUtils::isUtf8() referring to
  PHP 5.3, which is no longer supported in MediaWiki, to indicate
  that the same issue also exists in old versions of HHVM. (If
  we don't have to support 3.4 or older, then the function could
  be deprecated and removed if desired.)

Follows-up 943563062f.

Change-Id: I55e5cd534b849c6ea06a7fadacbbf34a12d87ebe
2016-04-07 09:02:37 -04:00
Reedy
83fb19cb13 Swap the rest of array() -> []
Change-Id: I76a7259ed952a0673a1941f08b39b545211fba07
2016-03-30 22:04:58 +00:00
Timo Tijhof
221829c4c1 MemoizedCallable: Add test for memoized closure
Follows-up da3443bfa2.

It won't work across different requests due to use of uniqid()
internally (so we may want to use a HashBagOStuff for closure
instead to save memory in APC). But at least assert that it does
and is intended to work.

Change-Id: Id4b42e8717f62f733eee8fcc9a4ab6c0f5c1f2fe
2016-03-08 16:42:21 +00:00
jenkins-bot
7c0213066b Merge "CachedBagOStuff: cache backend misses" 2016-02-23 15:03:42 +00:00
Timo Tijhof
4193700b19 Add missing namespace to @covers comments
PHP_CodeCoverage_Exception:
> Trying to @cover not existing method "SwiftFileBackend::sanitzeHdrs".
> Trying to @cover not existing method "LineFormatter::normalizeException".
> Trying to @cover not existing method "MonologSpi::mergeConfig".
> Trying to @cover not existing method "ProcessCacheLRU::het".
> Trying to @cover not existing method "BitmapHandler::swapICCProfile".
> Trying to @cover not existing class or interface "checkParseSafety".
> Trying to @cover not existing method "Article::__call". (was removed).
> Trying to @cover not existing method "ExtensionProcessor::extracttExtensionMessagesFiles".
> Trying to @cover not existing method "FileContentsHasher::getFileContentHash".

Makes code coverage run fail at the moment. These used to be warnings
in PHPUnit 3.x, but are now hard exceptions in PHPUnit 4.x when requesting
a coverage report.

Change-Id: If7f45ca57fd7d480d35b1414a889398837c0c472
2016-02-23 03:56:49 +00:00
Bryan Davis
2bcf34421a CachedBagOStuff: cache backend misses
Cache misses from the backend cache the same as hits.

Bug: T127772
Change-Id: If2fe1920411b24862acea888c627db13717da8bd
2016-02-22 18:45:20 -07:00
Timo Tijhof
5a3fcb0280 Timing: Fix flaky TimingTest test
> 1) TimingTest::testMark
> Failed asserting that 1455828402.4503 is greater than 1455828402.4503.

Use usleep(100), similar to the other tests already there.

Change-Id: Ic45c99b7e928df397809d872ab8e41ada6ecf1e7
2016-02-18 21:06:34 +00:00
Bartosz Dziewoński
1decdbce53 CSSMin: Make isRemoteUrl and isLocalUrl really private, now that we can in PHP 5.5
Well, protected, because we want to have unit tests for them, apparently.

Change-Id: I734b88599e5860aa59a07a89cc5389eb73b48813
2016-02-17 15:50:08 +01:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
addshore
be3176e75a Stop doing $that = $this in Tests
Closures support $this as of 5.4

Change-Id: Ibe73c789d093ef994b61bed23b4c76f7ef057a33
2016-02-09 23:27:25 -08:00
Brad Jorsch
25dbd91513 Clean up after Ie161e0f
Ie161e0f was done in a hurry, and so didn't do things in the best ways.
This introduces a new "CachedBagOStuff" that transparently handles all
the logic that had been copy-pasted all over in Ie161e0f.

The differences between CachedBagOStuff and MultiWriteBagOStuff are:
* CachedBagOStuff supports only one "backend".
* There's a flag for writes to only go to the in-memory cache.
* The in-memory cache is always updated.
* Locks go to the backend cache (with MultiWriteBagOStuff, it would wind
  up going to the HashBagOStuff used for the in-memory cache).

Change-Id: Iea494729bd2e8c6c5ab8facf4c241232e31e8215
2016-02-03 21:45:18 +00:00
umherirrender
54c1e18eec Remove various double empty newlines
The double empty newline is not needed between functions, variable or at
end of file

Change-Id: Ib866a95084c4601ac150a2b402cfa184ebc18afa
2015-12-27 18:55:12 +00:00
Aaron Schulz
5ffaa6ae0d Avoid "Unable to set value to APCBagOStuff" exceptions
* This can happen due to incr/add races. Use incrWithInit()
  instead to handle such cases.
* Also made BagOStuff:incrWithInit() return the new value like incr().

Change-Id: I0e3b02a4cff7c20544a9db2eaabd3f61e5a470b1
2015-12-21 20:41:43 -08:00
Aaron Schulz
ba06ccb394 Fix get()/getMulti() check key race condition in WANObjectCache
If a set() happened around the exact same time as check key was
initialized via get(), the curTTL in future get() calls could
sometimes be 0 instead of a negative value, even though hold-off
should apply.

Change-Id: Ide1fd65112aff425a4798e2ec406d71f2a8e84a7
2015-12-04 14:46:18 -08:00
Aaron Schulz
2ca76c62d0 Remove redundant testGetMulti() assertion
Change-Id: Iadbf39de9c6be2baeb7c2f7c1dce39e9f2bed2e4
2015-12-04 21:03:09 +00:00
Aaron Schulz
680f152715 Add 0 hold-off TTL support to WANObjectCache::delete()
This removes the peculiar >= 1 restriction

Change-Id: I255dc6ef9750ad2ed6234e3fcfed9ae22a038748
2015-11-23 21:46:36 -08:00
Gergő Tisza
10cf632683 Disallow associative arguments in ObjectFactory
There is no strong use case for associative "decoration"
of constructor arguments (the documentation benefits
mentioned in I43aa085 are outweighed by the confusion
caused by not failing loudly when someone passes an
associative argument by accident, e.g. by omitting an
array nesting level), so disallow them but make sure
they fail nicely, not with an invalid offset error.

Change-Id: I09e4af85ded6a1497b0db0265d2ee6707f91f5e3
2015-11-24 00:10:42 +00:00
Gergő Tisza
20b7696494 Support named arguments in ObjectFactory
Arguments are sometimes given as array( 'foo' => 1, 'bar' => 2 )
which makes the configuration self-documenting.

Change-Id: I43aa085090f1014ba841641867ebf9559d16e76d
2015-11-23 23:04:08 +00:00
Aaron Schulz
545ffd2d4c Fudge testGetWithSeveralCheckKeys() in both directions to be deterministic
Change-Id: I0f25cbd4555033f492b1b5d4ab55669240f19fc8
2015-11-20 14:21:31 -08:00
Timo Tijhof
a0cce5e4b6 objectcache: Implement check keys per cache key in WANObjectCache::getMulti()
To allow batch queries for multiple keys that themselves have different check
keys. Previously check keys always applied to all keys being retrieved.

Change-Id: I9e5ba198d79020ce05a802a510762e29fcfb2f1b
2015-11-17 04:15:33 +00:00
Aaron Schulz
6ae5dde077 objectcache: Add more WAN cache check key unit tests
Change-Id: If9bfef463b294391a456c5f04f48858f9dbe73fd
2015-11-17 03:28:18 +00:00
Timo Tijhof
6c3b45d4d3 objectcache: Add more @covers to WANObjectCache unit tests
Has only ~ 30% coverage according to PHPUnit, but mostly due to
lack of @covers annotations.

Change-Id: I871b45d8d8ee22c7cda644c450d78f577584e78c
2015-11-16 21:49:52 +00:00
Aaron Schulz
209217da8a objectcache: Make HashBagOStuff LRU instead of least-recently-set
Refresh key in get() in addition to just set().

Change-Id: I7985b98b6a346eaed8bf0a7349b95fabea8e8614
2015-11-04 01:30:39 +00:00
Ori Livneh
8340400f27 Add Timing interface
* Add `Timing`, an interface which mimics the W3C User Timing API.
  It provides a canonical way to store and retrieve markers (timestamps)
  and measures (timestamps + duration).
* As the initial use-case, use it to record 'requestShutdown'.

Change-Id: I36b29162ffcc091406df025463b0e2797e52f19a
2015-11-03 15:05:37 -08:00
Timo Tijhof
caa3e2ea06 objectcache: Add a clear() method to HashBagOStuff
Including tests for delete() and clear().

Change-Id: If39d729838e312523f0df3ae8b235ebe939a17fd
2015-11-03 10:07:10 +00:00
Timo Tijhof
4ec550812b objectcache: Refresh key in HashBagOStuff::set() for maxKeys eviction
* Match behaviour of MapCacheLRU and ProcessCacheLRU.
* Add missing unit tests for TTL and maxCacheKeys eviction behaviour.

Change-Id: I559eae1cd336274b21728e86775cfbad7e2f2c6d
2015-11-03 01:59:30 -08:00
Timo Tijhof
c7fc73286d objectcache: Move unit tests for HashBagOStuff to its own suite
Follows-up 7cddc22.

Also use PHPUnit_Framework_TestCase as parent instead of MediaWikiTestCase
in preparation for library extraction.

Change-Id: I0e68e56ecf8376b52a59c33ba6dd18b671bdcfc9
2015-11-03 05:37:28 +00:00
Aaron Schulz
7cddc22fb8 objectcache: Allow bounded HashBagOStuff sizes and limit it in WANObjectCache
Change-Id: Icca2474b1ea6feb7134f8958aecf79aa51b7f71e
2015-10-31 02:04:01 +00:00
Aaron Schulz
6d0108d533 Fix slow callbacks in getWithSetCallback() using lockTSE
* Keys that take a long time to generate would run into
  the MAX_SNAPSHOT_LAG check and have set() no-op. This
  would be fine except that leaves no key there to figure
  out the time since expiry and therefore whether to use
  the mutex, so it didn't. This now saves the keys but with
  a FLG_STALE bit set, making the next caller that sees it
  perform a regeneration (unless it can't get the mutex).
* Bumped LOCK_TTL so that keys can stay locked much longer.
* This is easy to test via sleep(5) in tagUsageStatistics()
  and two Special:Tags browser tabs.

Bug: T91535
Change-Id: I549e70ace3d2e7da40d3c4346ebacc36024cd522
2015-10-28 23:31:37 +00:00
jenkins-bot
33859dfcf5 Merge "Make WAN cache HOLDOFF_TTL smaller by combining db/snapshot lag" 2015-10-28 15:10:40 +00:00
Ori Livneh
e9d64a69bc Replace local RunningStat with wikimedia/running-stat library
* mediawiki/vendor change: I66b419c8168df.
* Add back-compat shim in includes/compat/RunningStatCompat.php.

Change-Id: Ibad4dc0ce5aa8697791865ed8ea61a25bae7b3e0
2015-10-27 19:16:58 +00:00
Aaron Schulz
05a3ee2468 Make WAN cache HOLDOFF_TTL smaller by combining db/snapshot lag
* In the common ~0 second lag case, transactions up to ~7 seconds
  long will have normal set() behavior (instead of just ~5 seconds).
* Like wise with ~0 second transactions tolerating ~7 seconds of
  lag (instead of just ~5).
* The lower hold-off time lets caching resume 3 seconds sooner.

Change-Id: I21e2a0a6915492cec422b6a6af5541937745c15b
2015-10-27 11:38:54 -07:00
Bryan Davis
fecb2320a8 ObjectFactory: avoid using ReflectionClass
Add a new ObjectFactory::constructClassInstance() method that uses
a loop unrolling type of technique to avoid using ReflectionClass when
creating new class instances with 10 or fewer constructor arguments.

I really wanted to also include the use of PHP 5.6's `...` splat
operator when supported but there is no way to conditionally use a new
operator in a way that still allows older versions of PHP to parse the
same source file.

Bug: T115729
Change-Id: Ia29c4526f4bac51696654c9b0677cb3f70359966
2015-10-25 00:13:32 +00:00
Aaron Schulz
cce813a922 Move MultiWriteBagOStuff to /libs
Also moved related tests files to /libs.

Change-Id: I806eeaa30205733d497adde933baf0c4157f7aae
2015-10-24 12:15:26 -07:00
Ori Livneh
da3443bfa2 Add MemoizedCallable for APC-backed function memoization
Add a simple class to `libs/` for memoizing functions by caching return values
in APC. I decided not to make this an external library just yet because I see
this as potentially a part of a larger functional programming library.

Doesn't use APCBagOStuff for two reasons: (1) avoid dependency on MediaWiki
code; (2) ability to pass third &$success parameter to apc_store, to
distinguish between cache misses and cached false values.

Use this in ResourceLoaderFileModule to cache CSSMin::remap.

Change-Id: I00a47983a2583655d4631ecc9c6ba17597e36b5f
2015-10-06 23:35:03 +00:00
umherirrender
eca8c2a72f Rename ObjectFactoryTest_Fixture to camel case name
Change-Id: Ib909c4f694b9774a0983abbb23bfd38ed8c05d5b
2015-10-03 21:29:17 +02:00
Amir E. Aharoni
c50cbe48ab Fix remaining Generic.Files.LineLength failures
Except the ones for which there is no clear way to fix
because of T114213.

Bug: T102614
Change-Id: I94a1fc7c890c82a36f0b51e42d311585c275775c
2015-10-03 17:23:07 +00:00
Reedy
8882634112 Re-enable MediaWiki.WhiteSpace.SpaceAfterControlStructure.Incorrect
Change-Id: I1b756909a1080108c5147d46950f826871ef6dc1
2015-09-26 18:32:32 +01:00
Timo Tijhof
2d9324aa8f IEUrlExtension: Add tests for the main use case the lib exists for
There are plenty of tests for the basic behaviour of the extension
detector and how IE favours the extension from the first question mark
segment that has a dot in the next segment.

But it was missing a test for the main use case the library exists
for, which is extensions in query strings when there is a file name
with extension already (but the dot is urlencoded).

Change-Id: I0f2e1387ae42da68020f1d793bb9eb5a87a06f24
2015-08-28 13:30:31 +02:00
Ori Livneh
e8c17972dd Replace bundled IPSet library with composer dependency
Complete the 'librarization' of IPSet by replacing the code in core with a
dependency on the external library.

Change-Id: I789b4fb42ee1da44ea3d8e1db551b047e11a439e
2015-08-26 10:38:09 -07:00
jenkins-bot
4590c637f5 Merge "CSSMin: remove dot segments in relative local URLs" 2015-08-20 14:11:36 +00:00
jenkins-bot
8ce0461710 Merge "CSSMin: version URLs based on content, not mtime" 2015-08-17 22:55:07 +00:00
Ori Livneh
3063e85248 CSSMin: version URLs based on content, not mtime
The content of these files is more stable than their mtimes, which change
every time we roll out a new branch. Because MD5 avalanches well[0], using
the first five hexadecimal digits is sufficient to ensure that the chance
of two successive versions colliding is improbably small (roughly one in a
million).

[0]: https://en.wikipedia.org/wiki/Avalanche_effect

Change-Id: I1bdf94c58786d2545311b238476b48217a5a60af
2015-08-17 15:22:07 -07:00
Ori Livneh
5844ebcd93 CSSMin: remove dot segments in relative local URLs
Call wfRemoveDotSegments() (if available) on relative local URLs, so that
relative dot segments are collapsed.

Change-Id: Icc1c23a2a7d96c869412965bd8c3730d3c3b3a53
2015-08-16 10:41:45 -07:00
Ori Livneh
3ecec5434e Only convert boolean true/false to !0/!1
Fix-up for I5ab29b686b8. If we encounter stupid code like
`a.true = 1;` or `a = { true: 1 }`, we should not convert that to !0/!1.
Because JSMin barfs on such input, it is necessary to add another parameter to
the test method which specifies whether or not the minified JavaScript is
supposed to be valid JavaScript by the standards of JSMin.

Change-Id: Ib78c628147fdb95982d6e33e0ab298584fb63d0b
2015-08-15 14:13:59 -07:00
Gergő Tisza
e56f7b6c63 Sample StatsD messages when instructed
Bug: T106457
Change-Id: I8ddb5a53dcbaf398f5a114d2a4a862842e980db4
2015-08-03 22:09:18 +00:00
Bryan Davis
e53600c4b6 ObjectFactory: add support for setter injection
Extend ObjectFactory::getObjectFromSpec() to support setter injection on
created objects when the specification includes a 'calls' member.

Bug: T107440
Change-Id: Ie2ece2e9658dd2d895d3935da4dc2da8a0a316e2
2015-07-30 11:24:07 -06:00
Kunal Mehta
4345629526 Improve appearance of libraries on Special:Version
* Added licenses
* Added description (will always be in English)
* Added authors
* Bold library name, like we do with extensions and skins

Change-Id: I880756b489fb5a3f81d398d32162a186a1b919d8
2015-06-26 21:26:05 +00:00
Vivek Ghaisas
fd9178e4c3 Fix braces code style
Bug: T102805
Change-Id: I7ce4c71dd605f9be79a630602391271bb269b962
2015-06-17 16:28:51 +03:00
Vivek Ghaisas
9f5b6f5aeb Fix whitespace issues around parentheses
Fix issues found by MediaWiki.WhiteSpace.SpaceyParenthesis sniff.

Bug: T102617
Change-Id: Iec7f71e64081659fba373ec20d9d2006306a98f4
2015-06-16 22:14:02 +03:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
Timo Tijhof
19bff2537e Consistently name the $this callback variable "$that"
Most are already, but some had differnet names. In PHP 5.4+ this is
redundant as they inherit automatically, but we need these for PHP 5.3
compatibility. Settle on "$that" to make these easier to find.

Change-Id: I0b68b2c550fde9c2da53d844421e34b8df0c42ed
2015-05-13 23:05:07 +01:00
daniel
7081228214 Start using the Assert helper class for checking parameters.
This introduces https://github.com/wmde/Assert as a dependency,
as discussed in the RFC T91071.

This change uses assertions to check some parameters in some places,
to showcase the main intended use case for assertions in MediaWiki.

Bug: T91071
Change-Id: I93ac39b7c146f10532e37b51d973b59b9c424b2f
2015-05-11 17:54:07 +00:00
jenkins-bot
e36a953cc2 Merge "tests: Clean up file headers" 2015-04-01 22:27:33 +00:00
csteipp
b7ee3aca38 SECURITY: Always expand xml entities when checking SVG's
XmlTypeCheck's use of xml_parse for filtering SVG's sometimes left xml
entities unexpanded, which can lead to false-negatives when the
callback was used for filtering. Update XmlTypeCheck to use XMLReader
instead, tell the library to fully expand entities, and rely on the
library to error out if it encounters XML that is likely to cause a DoS
if parsed.

Bug: T88310
Change-Id: I77c77a2d6d22f549e7ef969811f7edd77a45dbba
2015-04-01 09:56:20 -07:00
Timo Tijhof
b4bac102b6 tests: Clean up file headers
* Remove redundant @licence/@license from test suite files.
  They already have full licence headers. And @licence raises a
  warning in Doxygen.

* Fix weird messes of comments inside comments and other things.

Change-Id: I38da8ca76330f72b8dc22b0ecf1ea69d5ea55ede
2015-04-01 00:17:12 +01:00
Kunal Mehta
13975fe76a Use wikimedia/utfnormal library, add backwards-compatability layer
This drops support for the custom utf8 normal PHP extension in favor
of the intl extension.

Bug: T90825
Change-Id: Ifbaeb2ef684217cf6187ccc4fb4d303f89608300
2015-03-24 12:59:26 -07:00
Kunal Mehta
dfcca581bd Convert UtfNormalTest into a PHPUnit test
Change-Id: I20b90b1d9d2225d12adadb8c00c793fe7927f1bc
2015-02-20 06:04:26 +00:00
jenkins-bot
fd95d37add Merge "StringUtils: throw InvalidArgumentException and move into libs/" 2015-01-31 00:43:58 +00:00
jenkins-bot
7cfdac3aaa Merge "normal: Add @covers tag to UtfNormal test" 2015-01-29 20:55:13 +00:00
jenkins-bot
e5a5439f4f Merge "DeferredStringifier: Clean up comments and indentation in tests" 2015-01-29 20:54:53 +00:00
jenkins-bot
3a75e6a4e1 Merge "ProcessCacheLRU: Clean up doc comments and add @covers tags to tests" 2015-01-29 20:54:06 +00:00
Timo Tijhof
44c33d3d28 normal: Add @covers tag to UtfNormal test
Change-Id: Ib0d64bda44cbb1f72ea69025785442967bb26d74
2015-01-29 12:38:34 -08:00
Timo Tijhof
9f58d49a27 DeferredStringifier: Clean up comments and indentation in tests
Change-Id: Ifdf2fe711a0be495465255c6fabb55fe34b79b27
2015-01-29 12:10:08 -08:00
Timo Tijhof
9b62a6ad80 ProcessCacheLRU: Clean up doc comments and add @covers tags to tests
Change-Id: Ie7e8f4b31d89fc0bee748c03d94cb54b23ab4f83
2015-01-29 12:08:27 -08:00
Timo Tijhof
9a2e47d188 CSSMin: Add tests for isRemoteUrl and isLocalUrl
Change-Id: I5a84eb62eaac96d0dd7f7e27bf76f64e7d7657cf
2015-01-29 19:47:46 +00:00
Kunal Mehta
4675950b19 StringUtils: throw InvalidArgumentException and move into libs/
Bug: T87863
Change-Id: Iac5bd958c27cad834e35930d0b99adb75c60411b
2015-01-29 10:30:57 -08:00
Kunal Mehta
b2e1b70e15 Turn Utf8Test into a PHPUnit test case
Change-Id: I7e709f475a37a592fb3052f367894ebce32310f3
2015-01-29 00:08:01 +00:00
Kunal Mehta
c8900f30af Move includes/normal to includes/libs/normal
Change-Id: Ib530ad9dbe1d3a33dc53ef8b9620f61d4e1a2d62
2015-01-29 00:00:35 +00:00
Kunal Mehta
6a55db1a97 Move DeferredStringifier into libs, add tests
Change-Id: I384d1a3854e957315584d30ec58c48c02fee6a2c
2015-01-13 11:04:44 -08:00
gladoscc
2967766734 Rename testBug32548Exponent to testExponentLineBreaking
Fix for @todo

Change-Id: I0dcef36fe9cee95ae320b535bac883d5e26ad140
2015-01-09 12:06:31 +00:00
Kunal Mehta
17ccfbb7b3 Don't display composer installed extensions/skins on Special:Version
Extensions and skins should already be adding their credits through
$wgExtensionCredits, meaning that they would be duplicated in the
"Installed libraries" section.

Anything that has a type that begins with "mediawiki-" will now
be skipped.

Release notes were added for this commit and 528297f8db.

Change-Id: Ie873b16c1a8a696c7de833ae69b263161d9e15a8
2015-01-05 15:58:43 +00:00
Kunal Mehta
6739df6b7f Fix @covers tags for includes/libs/composer
Change-Id: I90a39bce22f304eb06dd595eb845bb953e6937c3
2015-01-02 15:03:43 +00:00
Chad Horohoe
59afd687ce Convert most libs/ tests to use PHPUnit_Framework_TestCase
They don't actually need any of the mess that MediaWikiTestCase provides

Change-Id: Ibd067480fc294096d1249132cb800e09889efb18
2014-12-29 20:53:24 -08:00
Kunal Mehta
ce49874d9f Add checkComposerLockUpToDate.php script
Checks whether your composer.lock file is up to date
with the current composer.json file.

Bug: T77388
Change-Id: I528d63172c238cf1ea9bc02e8eb39b93225865de
2014-12-29 23:20:30 +00:00
umherirrender
1d8b52fbfd Move Test files under same folder structure where class is (/includes/)
Change-Id: I95f1aa6f0ed2cc3306aa6e588a11f359854315c1
2014-12-16 21:56:44 +01:00
Timo Tijhof
ff6513f28b Remove '@section LICENSE'
This was used in 2 special classes, the logger classes and spread
to a few other random classes.

Afaik this has no meaning. Is for something we don't use, and
goes against the meaning of '@section' in Doxygen, which we do
use.

In Doxygen output, all LICENSE references became links to
ProfilerXhprof (the one Doxygen encoutered first).

Bug: T72328
Change-Id: Icc7c443245c70bc0f549bee7d105eef5691c864d
2014-11-26 02:20:55 +00:00
Kunal Mehta
987d2e4c77 Use cdb library and provide a back-compat layer
The new cdb library is pulled in via composer. Since the
library uses namespaces, a backwards-compatability layer
is provided for the old class names:
* CdbReader
* CdbWriter
* CdbException

The PHP/DBA-specific classes should never have been used directly.

Depends on I98302bdf1 in mediawiki/vendor

Change-Id: I39549ac8540b262cf91f7d1830d36327afb3033d
2014-11-19 18:39:27 +00:00
Bryan Davis
98a34a5224 Work around test provider running before setUp()
If the xhprof extension is not present we skip running the test, but
phpunit runs the test provider function before that check so we need to
guard against missing constants to avoid spurious warnings in the test
output.

Change-Id: I5541a062ff0c47ca8802315554b3f32dfd01dcd0
2014-11-13 13:18:08 -07:00
jenkins-bot
0e2a9baff3 Merge "Enable profiling via xhprof" 2014-11-12 22:00:30 +00:00
Kunal Mehta
e774a8a779 Add @covers tags to ObjectFactoryTest
Change-Id: I0130b133ef8261e43082d90b08cf2b9abf64ffc9
2014-11-12 17:17:36 +00:00
Kunal Mehta
0136b75d1a Move CdbTest into includes/libs/cdb as well
Change-Id: If0ad645129692129d5674fd0e6651032b03a13c0
2014-11-10 11:02:46 -08:00
Bryan Davis
46c47e11bc Enable profiling via xhprof
Add a helper class to assist in collecting profiling information using
XHProf <https://github.com/phacility/xhprof> and a Profiler
implementation to hook it into the existing MediaWiki profiling system.

The Profiler includes support for generating tabular reports similar to
the traditional output of ProfilerSimpleText and ProfilerSimpleTrace or
sending data to a udpprofile daemon as supported by ProfilerSimpleUDP.
It also produces data compatible with the debugging toolbar.

Bug: T759
Change-Id: I16a75cb7636cb5dcef3830d738b2dcd2047d0aaa
2014-11-08 11:47:25 -07:00
Kunal Mehta
d5563db924 Deprecate MWFunction::newObj() in favor of ObjectFactory
Change-Id: Iaa803311409cf7b649f64f69bafe2935a418d31c
2014-11-06 13:41:18 -07:00
Bartosz Dziewoński
bf3b97791b CSSMin: Don't generate double rules for IE < 8 when embedding SVG files
Bug: 71003
Change-Id: Ic232e8d62b164940003afdfe7cce9f964d7e9cbc
2014-09-30 21:19:58 +00:00
Bartosz Dziewoński
fa223d65d6 CSSMin: Do not base64-encode non-binary files when embedding
Do not base64-encode non-binary files (containing only whitespace and
printable ASCII characters, which matches sane SVG files).

For SVG files the percent-encoded URIs are actually slightly longer
than the base64-encoded ones (~10%), but compress a lot better
resulting on 15-20% less data to transfer after gzip compression.

(The effect is best seen on the 'oojs-ui' module, which consists
mostly of SVG icons – especially after commenting out everything other
than 'oojs-ui.svg.css'.)

I tried this for binary files too, just in case; but as expected, they
suffer from a noticeable size increase even with compression (~15%).

Bug: 67341
Change-Id: Iddaf863b6be98570a2bb8e606f13946a96345f65
2014-09-30 23:13:42 +02:00
Timo Tijhof
282847e6d3 Use CSSJanus from upstream (v1.1.0)
Remove the local copy and include upstream instead.

Remove local unit tests as it's now a proper upstream lib that
is tested by uptream.

Release:

  https://github.com/cssjanus/php-cssjanus/releases/tag/v1.1.0

Changes:

  Features
  * Support "/*!" syntax for @noflip.
  * Flip border-style.

  Bug fixes
  * Improve flipping of background-position.
  * Restrict four_notation_quantity to margin, padding and border-width.
  * Flip two values in border-radius rule (in addition to four values).

Change-Id: Ieb6e179d6163f99a9c98fd99c2277d25135871fe
2014-09-25 18:00:27 -07:00
Bartosz Dziewoński
4c01f8b2bc Make "/*@noflip*/ /*@embed*/" annotation work without CSSJanus hacks
This reverts most of commit 2d842f1425,
leaving only the test added in it, and reimplements the same
functionality better.

Instead of stripping /*@noflip*/ annotations in CSSJanus, which is
incompatible with other implementations that preserve it, extend
CSSMin to allow other CSS comments to be present before the
rule-global @embed annotation. (This required making the regex logic
in it even worse than it was, but it's actually slightly less terrible
than I expected it would be. Good thing we have tests!)

Bug: 69698
Change-Id: I58603ef64f7d7cdc6461b34721a4d6b15f15ad79
2014-09-23 22:47:54 +00:00
Timo Tijhof
96771e3a65 test: Clean up data providers that should be static
Follows-up b36d883.

By far most data providers are static (and PHPUnit expects them
to be static and calls them that way).

Most of these classes already had their data providers static
but additional commits sloppily introduced non-static ones.

* ResourceLoaderWikiModuleTest, 8968d8787f.
* TitleTest, 545f1d3a73.
  Odd unused method 'dataTestIsValidMoveOperation' was introduced
  in 550b878e63.
* GlobalVarConfigTest, a3e18c3670.

Change-Id: I5da99f7cd3da68c550ae507ffe1f725d31e7666f
2014-09-18 12:52:44 -07:00
Bartosz Dziewoński
2d842f1425 Make "/*@noflip*/ /*@embed*/" annotation work
To do it, just remove /*@noflip*/ annotations in CSSJanus after
we're done processing. They are not needed anymore and some obscure
interactions with CSSMin logic for preserving comments caused
`/*@noflip*/ /*@embed*/ background-image: url(…)` not to work
correctly (it would not be embedded).

This also requires us to always do CSSJanus processing, even when we
don't need flipping, to consistently handle the annotations.
I'm not entirely sure if this is worth it, but I still greatly prefer
doing it to documenting this stupid limitation. :)

Bug: 69698
Change-Id: I311b12b08b2dff9d45efb584db08cf4a11318f59
2014-08-18 17:40:51 +02:00
Shahyar
923ad97b3a Correct shadow flipping in CSSJanus
Was incorrectly assuming values between -1 and 1 were 0
Fixes an issue with mw-ui-input class in RTL mode which
makes use of box shadow.

Change-Id: I13e52467721bc2534e0a7c5245e3380ac43736ef
2014-07-30 23:14:32 +00:00
jenkins-bot
cfda511688 Merge "Skip 64 bit MWMessagePack tests on 32 bit machines" 2014-07-09 16:59:58 +00:00
Thiemo Mättig
ceb196fa9f Skip 64 bit MWMessagePack tests on 32 bit machines
It's not only Windows. I have an Ubuntu machine with a PHP setup
limited to 32 bit (which is totally fine, it allows me to develop
and always stay compatible to Windows and other 32 bit environments).

I don't see an other way than to skip these tests on a 32 bit
machine. The loss (PHP clips values larger than 2147483647 to
2147483647) does not happen in the code that is tested but in the
test setup.

Change-Id: Ie9a173c0a92ed84eaaea981a25ba130f2eca169e
2014-07-09 17:21:40 +02:00
Stephan Gambke
3b1b2b0c33 Fix CSSMin url() remapping when comments in CSS contain curly braces
Remapping of url()s was thrown off if the CSS contained comments
with curly braces. To fix, replace all block comments (except @embed
directives) with placeholders before url() processing and swap them
back in afterwards. Patch also includes tests.

Bug: 60077
Change-Id: If18a93c17ea9bcd529f6a664aa2dcc51d4a37f38
2014-06-30 09:14:10 +00:00
Trevor Parscal
3e2fd810d2 Revert "Fix "CSSMin url() value remapping not working in certain obscure cases""
This reverts commit fa1053a16a.

Breaks CSS output, making the skin and extensions look unstyled.

Change-Id: I297150f775dd5559f91f2c9c25c64b134459c6b6
2014-06-10 22:04:40 +00:00
Stephan Gambke
fa1053a16a Fix "CSSMin url() value remapping not working in certain obscure cases"
This patch extends the regex used to find URLs by css block quote subpatterns.

Bug: 60077
Change-Id: Ife00d01b78c6369f20ffc49d20aa58e354a61ee7
2014-06-09 23:50:28 +02:00
Bartosz Dziewoński
b8c88afe59 CSSMin: Add tests for handling existing data: URIs
Just a sanity check inspired by bug 66112.

Change-Id: Iac60a1eb15b3ff60822a3f24f2511c4bc64c092a
2014-06-05 18:38:41 +02:00
Brandon Black
e323bc3393 Speed up CIDR matching from $wgSquidServersNoPurge
This adds a new generic library class IPSet, which precomputes
a reasonably-efficient data structure from the input list of
addresses and CIDR net/mask specs for fast runtime matching,
and then uses it to check trusted XFF-setters.

See also: 32b4f19f173fc5cff1029eedee63a39a2d72dd3a
Bug: 57021
Change-Id: Ia3b12fb90c3e7e492374a128943b014481cc2730
2014-05-15 17:23:12 +00:00