Commit graph

222 commits

Author SHA1 Message Date
WMDE-Fisch
6df9ed1ad6 update mediawiki-codesniffer to 0.11.0 and fix issues
- mostly auto fixes
- some too long lines fixed
- ignore amp space in one case  passing by reference

Change-Id: I6472f83bc3cbf4bd629d83050cc3319b19ec465c
2017-08-11 22:27:51 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Thiemo Mättig
91a920fd85 Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".

Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
2017-07-21 12:19:30 +02:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
jenkins-bot
bc213a9d46 Merge "build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0" 2017-06-27 08:57:28 +00:00
Bryan Davis
aaf1e17088 Remove "@author Bryan Davis" and "Brad Jorsch" annotations
Undo traces of a practice we carried over from past projects and
existing examples that is neither universal nor actively encouraged in
the MediaWiki codebase.

Bug: T139301
Change-Id: I5c9c89b72a45a44aa4264a5e57b003c1a86cdf6e
Co-Authored-By: Brad Jorsch <bjorsch@wikimedia.org>
2017-06-26 15:25:46 -06:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
WMDE-Fisch
7b5f08e703 Replace deprecated Context::getStats() with MWServices::getStatsdDataFactory()
Change-Id: I1756f69ca2ebd301a5049bf758d1a87c37771fe6
2017-03-17 12:07:03 +01:00
James D. Forrester
9635dda73a includes: Replace implicit Bugzilla bug numbers with Phab ones
It's unreasonable to expect newbies to know that "bug 12345" means "Task T14345"
except where it doesn't, so let's just standardise on the real numbers.

Change-Id: I6f59febaf8fc96e80f8cfc11f4356283f461142a
2017-02-21 18:13:24 +00:00
Aaron Schulz
59053a0716 Move ChronologyProtector/TransactionProfiler to Rdbms namespace
Change-Id: I37a655bd8bd267c9bc32028b55925b2dce527d33
2017-01-26 10:30:57 -08:00
Kunal Mehta
de53d90568 profiler: Support tideways for PHP7 profiling
xhprof does not support PHP7, and it doesn't seem like upstream will be
working on that any time soon.

Tideways is a profiler that is basically a drop-in replacement for
xhprof with functions renamed. So Xhprof::enable() and Xhprof::disable()
will now try to use tideways if that is installed and xhprof is not.

Bug: T152186
Change-Id: I0d7d2de56ac638ca2851c662f527049bd620c0e9
2016-12-06 23:38:19 +00:00
Kunal Mehta
61adc1e146 Use namespaced ScopedCallback
The un-namespaced \ScopedCallback is deprecated.

Change-Id: Ie014d5a775ead66335a24acac9d339915884d1a4
2016-10-17 15:46:05 -07:00
Aaron Schulz
058707ef38 Fix SectionProfileCallback type error
Avoids "Argument 1 passed to Profiler::scopedProfileOut() must
be an instance of ScopedCallback, SectionProfileCallback given"

Change-Id: I92713de71df9722d8a5d7e5cd04460aff71cf096
2016-09-30 15:57:44 -07:00
Kunal Mehta
5320f0835e Use wikimedia/scoped-callback
The ScopedCallback class was moved into a separate library. This updates
all callers to use the namespaced version, and provides a
backwards-compatibility class wrapper under the old name.

Bug: T146258
Change-Id: I2dd0a66fe2f510f26bdfef6b0a975c1beb3fd93c
Depends-On: Iea0c40bdd7776372ccf72db8f088a2abaa4d3721
2016-09-29 10:41:40 +00:00
Aaron Schulz
531aa2906e Move TransactionProfiler to /libs/rdbms
Change-Id: I13173facc9f101bca69e3bf3e85f292f8468e855
2016-09-14 05:18:57 -07:00
Bartosz Dziewoński
a49b3216fc Canonicalise '@deprecated since' doc comments
Change-Id: I440041512feb47a998efb45e76e3fd00e0655752
2016-08-02 22:21:57 +00:00
Aaron Schulz
a2e3ae5e18 Silence the TransactionProfiler in masterRunningReadOnly()
Change-Id: I93a5cce8c781c1e2ab4fb6bcefd3720e61f8a4dd
2016-07-30 13:42:44 -07:00
Kevin Israel
dc529b6555 SectionProfiler: Remove $profileOutCallback
Unused since 626aede99b.

Change-Id: I12c0d7900cd81d8b33b1a5a8dbe866ddd9664083
2016-07-25 09:23:29 -04:00
Chad Horohoe
b0dad16ea9 ProfileStub: Remove getCurrentSection(), unused
No callers, no implementations in other Profiler classes

Change-Id: I5b5364d5b40964944e1595056b4c292610fcdf36
2016-07-20 20:48:28 +00:00
Ori Livneh
20abb707ba Decouple Xhprof profiling from profiling data processing
The motivation for this patch came from trying to use xhprof to profile the
unit tests. I was able to profile specific test suites, but if I tried to
profile a complete PHPUnit run, I ended up with empty profiling data. My
initial suspicion was that this was due to some Xhprof buffer getting
exhausted. The actual reason ended up being much simpler: the XhprofTest suite
indirectly called xhprof_enable() / xhprof_disable(), which stopped xhprof and
cleared out the data, so that when I was calling xhprof_disable() at the end of
the run, there was no profiling data to return, because xhprof was not running.

For the most part the XhprofTest was already doing the right thing by trying to
avoid having side-effects or relying on xhprof. Wherever possible, test fixture
profiling data was used in lieu of actually running xhprof. But this was not
totally successful because the Xhprof class coupled the collection of data to
the processing of data. Xhprof::__construct() called xhprof_enable(), so there
was no real way around that.

I think that the right way to fix that is to decouple profiling from profiling
data analysis. Thus I renamed 'Xhprof' to 'XhprofData', and modified the class
so that it expects to be fed profiling data rather than going out and
collecting it on its own. As a result, it is now possible to profile a full
phpunit run with xhprof, and the work that went into writing fixtures for the
Xhprof unit tests pays off: the class and the tests no longer have a hard
dependency on the xhprof extension, and the tests do not have to be skipped
when it is not installed. And the tests are really testing the system under
test, rather than the xhprof extension.

Finally, I added a new Xhprof class, which really is just an extremely thin
wrapper around xhprof_enable() / xhprof_disable(). The only extra functionality
it provides is the ability to check whether xhprof is running, via
Xhprof::isEnabled(). Calling Xhprof::enable() when it is already enabled will
cause an exception to be thrown. This should help us avoid running into
situations where two components contend for control of the profiler without
realizing it. A unit test tests this behavior.

The only part of this change that is not covered by tests is the change to
ProfilerXhprof. I tested it manually and it works.

Change-Id: Ica96beded68f04083abaf48ef1ae8c726eb60fa6
2016-05-12 22:23:05 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Timo Tijhof
487265cd17 Remove unused $wgStatsMethod and related UDPProfiler variables
All unused:
* $wgProfileCallTree
* $wgProfilePerHost
* $wgUDPProfilerHost
* $wgUDPProfilerPort
* $wgUDPProfilerFormatString
* $wgStatsMethod
* $wgAggregateStatsID
* $wgStatsFormatString

The only one still used was $wgProfilePerHost.

Also move release notes for ade945b97a from HISTORY to RELEASE-NOTES (Oops!)

Change-Id: Ia0280d23674ace1ac33aa561b06ee73cab865b29
2015-12-07 21:38:22 +00:00
jenkins-bot
1018d0a74b Merge "Make TransactionProfiler show the actual query times" 2015-10-23 01:32:00 +00:00
Aaron Schulz
38bb2dc30e Make TransactionProfiler show the actual query times
Also removed HTTP method from the message as logstash
gets those from the logger already

Change-Id: I7a6d9cb3ec8c7fcb53a081ee2bd055ff8c1c72c1
2015-10-21 23:30:23 -07:00
Aaron Schulz
660a775931 Remove old ProfilerOutputUdp class
This uses a non-standard output and requires a custom
collector that wmf does not maintain nor use anymore.

Change-Id: I41a68f7061465417fbdc5ca41f8eb6e1f99f1111
2015-10-21 20:59:47 -07:00
Ori Livneh
602a74af5d Compat fix for HHVM on non-Linux OSs
RUSAGE_THREAD is not POSIX; it's Linux-specific. HHVM now compiles on OS X,
and calling getrusage( 2 ) on OS X explodes with:

    exception : f_getrusage is not supported: RUSAGE_THREAD is not defined on
    this sytem [sic]

HHVM is being ported to Windows, too, where RUSAGE_THREAD is likewise missing.
So check that PHP_OS === 'Linux' before using it.

Change-Id: I6eac7d2bb4edfa1e697ae72f89299d0d5ac135f4
2015-09-23 01:55:57 -07:00
Ori Livneh
0a25d776ad Move StatsD key normalization from ProfilerOutputStats to BufferingStatsdDataFactory
I'm not sure why I stuck `normalizeMetricKey' in ProfilerOutputStats, because
the transformation it applies are suitable for converting any arbitrary string
into a StatsD-safe metric key. This patch moves the method to
BufferingStatsdDataFactory, which ensures it applies to all metrics logged
within MediaWiki, and not just the Profiler.

Supercedes If0237cdd0d.

Change-Id: I496ed748000d28f5399fee6e3cc271a1f68bd058
2015-07-13 19:07:02 +00:00
Aaron Schulz
033f71f3df Made text profiler output a bit less noisy by bumping "thresholdMs"
Change-Id: Ie66794c94fb3f6a3dc86ad604162b8e4f91c35ba
2015-06-26 07:35:28 +00:00
Jackmcbarn
c4f69827bc Fix some PHPCS issues
Change-Id: I6289ce362f8dfde7baf2b665f082820898844973
2015-06-26 01:32:28 -04:00
Ori Livneh
c099155a17 ellapsed => elapsed
Also fix some files that don't end with a newline.

Change-Id: Id0672d685b929a5832b42f733dad49683536180a
2015-06-23 03:32:33 +00:00
Vivek Ghaisas
fd9178e4c3 Fix braces code style
Bug: T102805
Change-Id: I7ce4c71dd605f9be79a630602391271bb269b962
2015-06-17 16:28:51 +03:00
Vivek Ghaisas
9f5b6f5aeb Fix whitespace issues around parentheses
Fix issues found by MediaWiki.WhiteSpace.SpaceyParenthesis sniff.

Bug: T102617
Change-Id: Iec7f71e64081659fba373ec20d9d2006306a98f4
2015-06-16 22:14:02 +03:00
Alex Monk
e7c9096ecc HTTPS-ify links to Wikimedia's sites in MW core source
See https://blog.wikimedia.org/2015/06/12/securing-wikimedia-sites-with-https/

Ignoring SVN stuff because svn.wikimedia.org's SSL certificate has expired, and
ignoring wgSharedUploadPath because T25122 wants to get rid of it.

Also a couple of BZ -> phab changes

Change-Id: I02fd23ac6c30a4a4c718e57d4dedbc693dd653c0
2015-06-13 19:23:23 +01:00
Aaron Schulz
016fd0b776 Refactored entry points to have uniform shutdown handling
* Added doPreOutputCommit() and doPostOutputShutdown(),
  which most entry points just using the later
* Also fixed problem where text profiling did not show up
* Avoid calling triggerJobs() in the file streaming
  entry points

Bug: T100127
Bug: T100085
Change-Id: Ibc7e768fd483389a01847f08cdeba4058c853d3f
2015-05-29 20:40:46 +00:00
Ori Livneh
e0ef4f556c ProfilerOutputStats: replace single ':' with '.', too
This maps 'hook: ' and 'query: ' to a metric name segment, instead of just a
metric name prefix.

Change-Id: I41fb71becbc862597b998c73ccfcaca5a3510ffc
2015-05-21 12:16:24 -07:00
Ori Livneh
6618307102 Improve ProfilerXhprof's blacklist/whitelist capabilities
* Apply the blacklist / whitelist to profiled sections, not just function names.
* Allow shell-style wildcard patterns in blacklist / whitelist.
* Prefix all profiled section names with 'section.', to distinguish them from
  functions.

Note that shell-style wildcard patterns are not supported by xhprof natively,
but it won't barf on them either, nor will they match against actual function
names (since shell wildcard characters are not valid for PHP function names),
and the filtering will still be enforced in ProfilerXhprof.

This has the side-effect of working around https://github.com/facebook/hhvm/issues/4385

Bug: T99829
Change-Id: I8354ed922fa7b42857eda03be8f62b89ac78d0d6
2015-05-21 12:09:12 -07:00
Ori Livneh
a70b4c85a8 ProfilerOutputStats: allow a key prefix to be specified
If one wants to nest all metrics emitted by the profiler under a metric
namespace, one can now set the 'prefix' param.

Task: T66301
Change-Id: I6c52f20e39017f4c818ca6623bb7f48683fc8abc
2015-05-21 11:11:21 -07:00
Ori Livneh
dfa5dabd15 ProfilerOutputStats: tweak stat key normalization
When we convert a profiler entry to a statsd metric key, we replace
nonalphabetic characters with slashes. When part of the key is a file path with
a leading or trailing slash, this can result in keys that look like:
'MediaWiki._srv_mediawiki_foo_.bar', which is a bit ugly. This patch trims
leading and trailing underscores from each dot-separated key segment.

Change-Id: I71172b956f5a6120a75981a15dedf372d3893b88
2015-05-14 14:49:51 -07:00
Aaron Schulz
aed94cf111 Include transaction details log entry for trx write time violations
* This makes it possible to see what is being done by matching
  the trx IDs of the (normally) adjacent log entries

Change-Id: I515de99acfe159856930d8e259c00eb744f43213
2015-05-08 19:17:49 +00:00
Aaron Schulz
7936b94341 Log transaction that spend too much time in write queries
Change-Id: I9ba184601cfb1667291f7a29677cc5205344d5ad
2015-05-07 17:13:20 -07:00
Aaron Schulz
a2ea6116af Fixed some <code> tags for doxygen
Change-Id: I550b8760556e4916ee9b63ee244bbbcb4d926142
2015-04-29 06:01:59 +01:00
Aaron Schulz
7ea13643f5 Added $wgTrxProfilerLimits and slow query limits
* Limits are now configurable instead of being hard-coded

Change-Id: I99133586eb82e8e9e84061548c8d1a99695fde5c
2015-04-28 10:18:11 +02:00
Chad Horohoe
ccf3d67934 Remove useless +x flag on ProfilerSectionOnly
Change-Id: I8f5190c114b50ee3b6288ce383caba7988c49895
2015-04-24 09:21:32 -07:00
Chad Horohoe
4b619eb7bd Remove $wgRUstart, unused
Change-Id: Ia57f8fb2da4eed6b185ea0592d521e3119411f0e
2015-04-16 15:28:37 -07:00
Gilles Dubuc
7b2d50e761 Track request method in dbperformance.log
This will allow us to avoid needlessly investigating master requests that
only happen on POST (which is already what we want for multi-DC).

Bug: T92357
Change-Id: Ia7437d00f5b89a8e318d85659d60e2f9f9f26149
2015-04-06 15:13:03 +00:00
jenkins-bot
497189d0a6 Merge "Use structured logging/MWLoggerFactory for TransactionProfiler" 2015-04-02 21:20:54 +00:00
Ori Livneh
27b9c0639d Simplify profiler output class selection
Instead of maintaining a mapping of short names to class names ('db' =>
'ProfilerOutputDb', etc.), let us adopt the convention of using the full
class name to indicate the output type. We can maintain backward-compatibility
by using simple string manipulation to transform short names to the full class
names.

Change-Id: I976e0da2873d88b9892fb41823cfe3af0a2d3974
2015-04-02 11:01:52 -07:00
Ori Livneh
4e5d016e72 Follow-up to Icf644ad34: Introduce ProfilerOutputStats
Change-Id: Ib3585303b75899c4cd7c9c88fb3473b441e52c23
2015-04-01 21:09:20 -07:00
Ori Livneh
1b6f70089d Introduce ProfilerOutputStats
* Associate Profiler objects with a request context by adding a $context
  property with a getter and a setter.
* Introduce ProfilerOutputStats, which writes profiling data to the stats
  buffer associated with the current request context.
* Make it the Profiler class's responsibility to enforce $wgProfilerLimit.
* Deprecate $wgProfilerLimit in favor of the (more aptly named, IMO)
  $wgProfiler['threshold'] config setting.
* Tidy up Profiler instance creation code in Profiler::instance().
* Add Profiler::getOutputs, which returns an array of ProfilerOutput instances
  which are configured for the current profiler and whose canUse() method
  returns true.
* Make ProfilerStub not log by creating a stub ProfilerStub::logData() method
  which does not call the parent. Previously the parent class checked if $this
  was an instance of ProfilerStub and returned early if so.

Task: T90623
Task: T85641
Change-Id: Icf644ad3435c1f30d0a49957a97b481808a3153d
2015-04-02 01:32:46 +00:00