Commit graph

91 commits

Author SHA1 Message Date
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
Brad Jorsch
71796c0e04 benchmarks: Fix divide by zero in Benchmarker
Follows-up 593442683d.

Bug: T186269
Change-Id: I011a8614e0acf489daf5bd9dcb932e920b995092
2018-02-02 02:24:49 +00:00
Timo Tijhof
5d42fb6549 benchmarks: Add benchmarkSanitizer.php
Covering the following common methods:

* validateEmail (input=valid, input=invalid)
* encodeAttribute (input=simple, input=special)
* safeEncodeAttribute (input=simple, input=special)
* removeHTMLtags (input=small, input=large)
* stripAllTags (input=small, input=large)

Change-Id: I3c645e960607ab247bd30324a2e70eb5fbcc5d6e
2018-01-28 13:49:48 -08:00
Timo Tijhof
593442683d benchmarks: Add rate per second and standard deviation to output
* Convert to using RunningStat to ensure mostly constant memory
  usage, instead of storing each result.
* Remove 'median' (no longer possible with this setup).
* Remove 'min' from output.

Change-Id: I6efbc500181eb502c1800165870eee81dbc418a7
2018-01-28 13:49:00 -08:00
Timo Tijhof
1e7b054f48 benchmarks: Simplify benchmarkLruHash.php by using a --method option
Easier to extend.

Change-Id: Ib9c6b3d10caad7de138d8bafd2f0e6e9bbe6a8cc
2018-01-28 13:48:48 -08:00
Timo Tijhof
be15056c37 benchmarks: Improve readme and convert to Markdown
* Add sections.
* Rename to *.md.
* Document that taskset is Linux-specific.

Change-Id: If6c5e54eda4dd1d6ea02445efa73e356dbc49016
2018-01-26 16:31:54 -08:00
Umherirrender
ad776c7d5f Use ::class to resolve class names in maintenance scripts
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
2018-01-23 17:40:16 +00:00
Umherirrender
255d76f2a1 build: Updating mediawiki/mediawiki-codesniffer to 15.0.0
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable

For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore

Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
2018-01-01 14:10:16 +01:00
Timo Tijhof
25ce819e1c maintenance: Add unit test for Benchmarker class
Change-Id: I3f835fc07741fd42b8adb5b019f72589429cc14f
2017-12-27 22:15:21 +00:00
Bryan Davis
9e34eeff23 Maintenance: add fatalError() method
Deprecate the second argument to Maintenance::error() in favor of a new
Maintenance::fatalError() method. This is intended to make it easier to
review flow control in maintenance scripts.

Change-Id: I75699008638f7e99b11210c7bb9e2e131fca7c9e
2017-11-21 21:34:16 -07:00
Roan Kattouw
a4631b92e7 Improve encoding of embedded SVGs
Unencode spaces, slashes, colons and equals signs.

Bug: T175318
Change-Id: Idebdfca8f93d5e090deba5bf5a256d7054e2d6c8
2017-09-14 11:39:57 -07:00
Timo Tijhof
3bcc039e7a benchmarks: Add benchmarks for MapCacheLRU and HashBagOStuff
Currently, HashBagOStuff leads in most benchmarks, except for
the --fill benchmark, where MapCacheLRU is faster on HHVM.
(by invoking with `PHP=hhvm mwscript maintenance/bench...`)

Run in MediaWiki-Vagrant (Debian Jessie) on MacBook Pro host.

$ benchmarkLruHash.php --count 50000 --construct
Running PHP 5.6.30-0+deb8u1:
- HashBagOStuff-construct (22% faster)
  total: 446.20ms (min: 0.01ms, median: 0.01ms, max: 2.09ms)
- MapCacheLRU-construct
  total: 575.31ms (min: 0.01ms, median: 0.01ms, max: 3.51ms)
Running PHP version 5.6.99-hhvm:
- HashBagOStuff-construct (13% faster)
  total: 124.70ms (min: 0.00ms, median: 0.00ms, max: 1.27ms)
- MapCacheLRU-construct
  total: 143.76ms (min: 0.00ms, median: 0.00ms, max: 2.01ms)

For MapCacheLRU, the main slowdown is its use of Wikimedia\Assert.
Removing that would make it faster than HashBagOStuff.

$ benchmarkLruHash.php --count 3000 --fill
Running PHP 5.6.30-0+deb8u1:
- HashBagOStuff-fill (22% faster)
  total: 10196.82ms (min: 2.91ms, median: 3.24ms, max: 9.51ms)
- MapCacheLRU-fill
  total: 13197.13ms (min: 3.92ms, median: 4.31ms, max: 9.90ms)
Running PHP version 5.6.99-hhvm:
- HashBagOStuff-fill
  total: 5700.37ms (min: 1.71ms, median: 1.85ms, max: 8.24ms)
- MapCacheLRU-fill (11% faster)
  total: 4986.90ms (min: 1.48ms, median: 1.62ms, max: 6.93ms)

Change-Id: Icd03f872dddb308f162c72674c8d2aa6092395e5
2017-07-27 21:40:42 +00:00
Timo Tijhof
0df8f6d49c benchmarks: Add benchmark for JSMinPlus
Example usage

> mwscript maintenance/benchmarks/benchmarkJSMinPlus.php --file resources/lib/jquery/jquery.js -v

Bug: T31784
Change-Id: I3c11574adb864a4bfbfceba137e285fdb8dc66d2
2017-06-27 22:17:39 +00:00
Timo Tijhof
cfb1379b9f benchmarks: Add reporting of resource usage
Change-Id: Ied06766cd02b14d835ed4217d5483f7cfad15af3
2017-06-15 18:21:53 +01:00
jenkins-bot
99da289757 Merge "benchmarks: Convert benchmarkHooks to use Benchmarker" 2017-05-11 21:19:11 +00:00
Timo Tijhof
0ec9359653 resourceloader: Add CSSMin benchmarks
Usage:

 # Run default benchmark
 $ php maintenance/benchmarks/benchmarkCSSMin.php

 # Use custom file
 $ php maintenance/benchmarks/benchmarkCSSMin.php --file resources/lib/qunitjs/qunit.css

 # Debug the output for inspection (no benchmark)
 $ php maintenance/benchmarks/benchmarkCSSMin.php --out

Change-Id: I70d118131d0ff16d1a811b2de1328ea622b7ca69
2017-05-04 04:13:42 +00:00
Timo Tijhof
74aca41874 benchmarks: Convert benchmarkHooks to use Benchmarker
Also change benchmarkPurge to *not* use Benchmarker since it
doesn't use bench() or any other Benchmarker method.

Change-Id: I5d8ace161ecf1e05d69abf2f242e3684ffe48fa0
2017-05-03 20:19:32 -07:00
jenkins-bot
9e398f813a Merge "Add benchmarkTidy.php, to benchmark tidy drivers" 2017-04-22 02:32:34 +00:00
Tim Starling
448be2ed3e Add benchmarkTidy.php, to benchmark tidy drivers
Plus representative input file

Change-Id: I254793fc55c57a98c07ae1e4c27e6005965c9a20
2017-04-21 01:02:22 +00:00
Timo Tijhof
fa3f8e1fd5 benchmarks: Report more metrics (min/max/median)
Add minimum, maximum, median to the report in addition to the mean (average)
which was already there. Based on benchmarkTidy.php from I254793fc5.

Example output:

> Delete
>    times: 10
>    total:   7.47ms
>      min:   0.53ms
>   median:   0.74ms
>     mean:   0.75ms
>      max:   1.21ms
>
> Truncate
>    times: 10
>    total:  72.38ms
>      min:   1.37ms
>   median:   8.32ms
>     mean:   7.24ms
>      max:  15.73ms

Change-Id: Ifd3064a3621e07f55505490403189cb47022c6c7
2017-04-20 04:54:32 +00:00
Timo Tijhof
dcc8b7e4ce benchmarks: Add setup, bench naming, and custom count default
* bench(): Add support for setup function.
  Demonstrated by converting bench_delete_truncate.php to use Benchmarker.

* bench(): Allow benchmarks to be named. Default remains (fn + args).
  Useful for closures.

* Benchmarker: Support overriding the default count of 100.
  Demonstrated in bench_delete_truncate.php to run 10x instead of
  100x (previous: 1x).

Change-Id: Iac182eaf3053f5bf0e811cd23082f530629d8a4e
2017-04-20 04:49:36 +00:00
Timo Tijhof
27822f0fa1 benchmarks: Minor clean up
Change-Id: I446ae1a9d9cdb6b26a6bb62367a432cea082f343
2017-04-20 04:47:18 +00:00
Aaron Schulz
21e71e0235 Use IDatabase type hints in /maintenance
Relatedly, move lockTables()/unlockTables() to IMaintainableDatabase

Change-Id: Ib53e9fa948deb2f9a70f0ce16c002613d0060bf9
2017-04-07 23:37:41 +00:00
James D. Forrester
242df680ce maintenance: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

This includes renaming fixBug20757.php to fixT22757.php for similar consistency.

Change-Id: If81a590d658fbd82c20c54ac47dfdc8856745ca3
2017-02-21 18:32:44 -08:00
Aaron Schulz
30f4b3c103 Replace DatabaseBase => Database in more places
Change-Id: If37a7909056bf2c31a8228cbc84f0fbbf5f1c517
2016-09-28 15:53:02 -07:00
Kunal Mehta
0efe030cd9 benchmarkParse: Add option to clear link cache
For performance testing the lookup of links (e.g. LinkHolderArray),
allow resetting the LinkCache after every parse.

Change-Id: I75c2cad9a8cf1b41d192708773194f799673ce83
2016-09-14 23:57:16 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Kevin Israel
5b48bf1b92 Clean up after "Kill mbstring fallbacks"
* Removed fallback code from Language, the associated data file
  (Utf8Case.ser), and the code to generate that data file.
* Removed comment in LanguageFi that "mb_substr has a compatibility
  function in GlobalFunctions.php".
* Removed check for mbstring in bench_utf8_title_check.php.
* In the tests for StringUtils::isUtf8():
  * Removed separate test for the non-mbstring code path.
  * Removed mentions of mbstring from function names and assertion
    messages, since mb_check_encoding() is now always used.
* Also updated the comment in StringUtils::isUtf8() referring to
  PHP 5.3, which is no longer supported in MediaWiki, to indicate
  that the same issue also exists in old versions of HHVM. (If
  we don't have to support 3.4 or older, then the function could
  be deprecated and removed if desired.)

Follows-up 943563062f.

Change-Id: I55e5cd534b849c6ea06a7fadacbbf34a12d87ebe
2016-04-07 09:02:37 -04:00
umherirrender
8678e32b8d Change remaining places to use short array syntax
Change-Id: I0785c73a239c11188ad7411c5cd8ed09a2dbc9b3
2016-03-11 18:41:26 +01:00
Siebrand Mazeland
5b119a0e44 Replace uses of join() by implode()
All of core uses implode() consistently now.

Change-Id: Iba50898c64c43f356d1caf8869f484e90d9ff651
2016-03-08 18:24:16 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Max Semenik
59db24e90b Use addDescription() instead of accessing mDescription directly
Change-Id: I0e2aa83024b8abf5298cfea4b21bf45722ad3103
2016-01-30 01:28:32 -08:00
Reedy
44cebea941 Update wfGetDB calls in Maintenance scripts to use getDB()
Change-Id: I9ad6745d84506b736dae94747256caac89715899
2016-01-02 16:58:23 +00:00
nicoco007
2c66553ed3 Added PHP/OS info to benchmark output
Added PHP version and architecture and OS version to benchmark output.

Change-Id: I018e6bf8bb60f594ac512ef2113e404dd304786a
2015-12-29 01:16:13 +03:00
Aaron Schulz
282c5fa9f3 Rename SquidUpdate => CdnCacheUpdate
Squid is not the only possible CDN

Change-Id: Ie2a2955847c5706e630322bbbab71c9d063b378f
2015-12-09 16:31:17 -08:00
Reedy
5d032c0f65 Rename wfBaseConvert.php maintenance script
Change-Id: I3174ceb040e9029ad8049a3e49c119bfac4aa386
2015-11-24 23:34:21 +00:00
Reedy
00c426e3c2 Replace wfBaseConvert with Wikimedia\base_convert
Change-Id: Iadab3d018c3559daf79be90edb23d131729bdb68
2015-11-24 22:51:42 +00:00
Chad Horohoe
f94e85e976 wfRunHooks() -> Hooks::run() in maintenance scripts
Change-Id: I6550bca3a6a3a6cd67bb1ed94c09805deed86dbc
2015-04-27 21:42:17 -07:00
Chad Horohoe
c33f4de066 Profile all external HTTP requests from MW
Change-Id: Ie980b080da2ef21ec7d9fc32f1accc55710de140
2015-03-03 20:54:30 -08:00
Max Semenik
1120bbe202 benchmarkParse.php: add options for HHVM testing
--warmup makes a specified number of loops before measuring;
--loops repeats parsing given number of times

Change-Id: I18bff702fce1f97c4afc6c9fd618fdc1f3272732
2014-07-15 17:23:27 -07:00
Siebrand Mazeland
f2f31fadc7 Update formatting in maintenance/ (2/4)
Change-Id: I2b791d3bff0de464b6bdaaeae0622c065389c31c
2014-04-23 20:08:42 +02:00
Siebrand Mazeland
c331f11991 Update formatting in maintenance/ (1/4)
Change-Id: I0be8111b6ff1feb9f3a4096a96a9c6a6e40feb05
2014-04-23 20:08:06 +02:00
Siebrand Mazeland
919c471829 Pass phpcs-strict on maintenance/benchmarks/
Change-Id: I73558930909b35bbc13d0a0ac9368485cb2aa8c4
2014-04-22 21:58:25 +02:00
umherirrender
e78776373e Fixed some @params documentation (maintenance)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I727deec35a712de0f0c676cc87dfa661f1ee965b
2014-04-17 22:48:32 +02:00
umherirrender
23fab68274 Fix spacing after @param and friends in comments
Searched for:
\@(param|return|throws|since|deprecated|access|todo|var)[ \t]{2,}

Change-Id: Icce22ba9fe0635455691ca58d9872d618151f346
2014-04-05 20:02:29 +00:00
umherirrender
047c86f26e Fix spacing between two functions
Added and removed some new lines to have one new line between two
functions

Change-Id: I1ccfbd575dd26b160396ef3d3e2e079f5cdbe196
2014-03-15 20:57:23 +00:00
Tyler Anthony Romeo
58a5641415 Code, style, and doc fixes for benchmarkParse.php
Fixes database query semantics, various documentation
issues, and unused variables.

Change-Id: Ic2c2d4d3912a9cd6703f7690f9d08df8ea08cd5c
Follows-up: I3bf3366a5ff58942 (dfd481e113)
2014-02-04 22:00:37 -05:00
Tim Starling
dfd481e113 Maintenance script for benchmarking parse operations
Including a feature allowing templates from some other time to be used,
which allows comparative benchmarking before and after the migration of
a template to Lua.

Change-Id: I3bf3366a5ff589421f6c52e9186e0cea05e6cff3
2014-02-05 00:40:21 +00:00
umherirrender
f153998317 Fixed spacing
- Removed double spaces
- Added space after if/switch/foreach
- Removed space on elseif
- Added space around parentheses
- Added newline at end of file
- Removed space before semicolon at end of line

Change-Id: Id40b87e04786c6111e6686d7f7eea1e588bdf37d
2013-11-19 19:03:54 +01:00
Antoine Musso
4d74280891 benchmarks: README file having run recommendations
In a wikitech thread about benchmarking, Tim Starling has been running
benchmarks using CPU binding and raised priority. This patch copy paste
Tim recommendations so we do not forget about them.

Change-Id: Ic6f2803c005f70a2f0400f28fafc3c8d209e0667
2013-06-26 11:55:57 +00:00