Commit graph

111 commits

Author SHA1 Message Date
James D. Forrester
4f2d1efdda Coding style: Auto-fix MediaWiki.Classes.UnsortedUseStatements.UnsortedUse
Change-Id: I94a0ae83c65e8ee419bbd1ae1e86ab21ed4d8210
2020-01-10 09:32:25 -08:00
Aaron Schulz
f72ae0f6e6 Switch various LoadBalancer::getConnection() callers to getConnectionRef()
This is the preferred method as it enforces read-only mode for DB_REPLICA
and handles LoadBalancer::reuseConnection() calls automatically.

Change-Id: Iab9439ba8e0810fa14c302661ed7a3534f6bfc0d
2019-07-12 10:56:30 -07:00
Umherirrender
36f9f13a9e Fix caller name in SiteStats::numberingroup/pagesInNs
Seeing {closure} in the logs as caller is not helpful

Change-Id: I0102cddc447257319fe8db7f8cbd837838653e99
2018-09-30 15:41:19 +02:00
Aaron Schulz
6535091de2 Salvage site_stats row with negative values in miser mode
* Instead of returning all zeroes, just use zero for the
  negative values in the row.
* Allow large numbers since the fields are BIGINT.
* Clean up the return types to truly be integers.
* Respect the $groups argument in SiteStatsInit::getDB().

Bug: T186947
Change-Id: I51fdc45124c12aba114540fc0ec66a3e63d61e09
2018-02-14 15:37:55 -08:00
Aaron Schulz
6094897687 Various cleanup to SiteStats classes
* Mark method visibility
* Remove unused code/methods
* Use WAN cache process cache in pagesInNs()
* Remove "m" prefix from SiteStatsInit fields
* Avoid use of $wg variables

Change-Id: Iab4001f02c9b2e8667ca4bac033fd4f6ef272148
2018-02-12 15:09:19 -08:00
Aaron Schulz
de2fe331aa Move SiteStatsInit to its own file
Change-Id: Ief4e5f83d8dce93641623c2fa9c2195a564ee656
2018-02-12 19:55:43 +00:00
Aaron Schulz
fcc7e1c028 Make SiteStatsInit::doPlaceholderInit() use 1 for ss_row_id
This makes it consistent with refresh() and avoids having two rows
on new wikis. Also make the SELECT explicitly look for row 1.

Bug: T186947
Change-Id: I4f952888bf8fecc791366a9698e46d61a4ad4ff3
2018-02-12 18:08:10 +00:00
Aaron Schulz
3f05f346a8 SiteStats row initialization cleanups
* Remove unreachable code (field cannot be both null and -1)
* Avoid PHP warnings during test runs in miser mode due to
  the $row field being "false".
* Init the site_stats row in unit tests to unbreak miser mode.

Change-Id: Ieb45cb3068b71e01aa12c674e0bfcf13a7f2493c
2018-02-02 08:22:03 +00:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
jenkins-bot
313dde5d06 Merge "Add WAN Cache to SiteStats::jobs" 2017-07-25 18:30:58 +00:00
Antoine Musso
29aaac5c63 Add WAN Cache to SiteStats::jobs
The method hits the jobrunner backend to find out how many jobs are
enqueued in each of the JobQueue.  It is publicly available via the
MediaWiki API request:
    /w/api.php?action=query&meta=siteinfo&siprop=statistics

That is often used by bots when querying recent changes among other and
with fast bot cause useless queries toward the jobrunner backend.

Wrap SiteStats::jobs() with a WAN cache under key SiteStats:jobscount.
Drop SiteStats::$jobs private variable that was used for in process
cache. The WAN Cache does it for us via 'pcTTL'.

That is similar to SiteStats::numberingroup().
Set TTL to one minute, which should still give fresh enough results for
public uses.

Cover that behavior with a test.

When writing tests I noticed MediaWikiTestCase generates a few jobs due
to the creation of the UTPage page:

* HTMLCacheUpdateJob to refresh backlinks (eg: history)
* RecentChangesUpdateJob which happens randomly

Pass EDIT_SUPPRESS_RC to doEditContent to prevent the first and blindly
delete entries in the recentChangesUpdate jobqueue for the second.

Change-Id: I95a272d0691d779bfee9e7a671cbab66a113dfa1
2017-07-25 18:17:34 +00:00
jenkins-bot
1a81ba9bcb Merge "SiteStats::jobs fix when there is a single job" 2017-07-24 20:40:53 +00:00
Thiemo Mättig
91a920fd85 Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".

Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
2017-07-21 12:19:30 +02:00
Antoine Musso
e3f7dde410 SiteStats::jobs fix when there is a single job
wfSpecialStatistics() used a database to store the jobs. When estimating
the number of row in the `job` table, even when there is no row MySQL
would still do a single row read. Hence it erroneously returned 1.

That got fixed in 7cd0b2e0b (T12228) with a hack.

Nowadays, the functionality has been moved to SiteStats which is backed
up by a JobQueueGroup.  It now returns the exact number of jobs and the
hack is no more needed.  We failed to remove it when porting it to the
JobQueue system.

Change-Id: I6ef55b3dcff3e47a92e2eeeddcbae078654f815a
2017-05-29 21:32:50 +02:00
Kunal Mehta
24ad91a622 SiteStats: Avoid deprecated wfMemcKey()
And the deprecated ObjectCache::getMainWANInstance() while we're at it.

Change-Id: If9d9ab9c661c003a3aa6c2d956700b1870a52f41
2017-05-25 11:20:23 -07:00
This, that and the other
f8008a7cb8 Remove $wgDisableUserGroupExpiry feature flag
This feature flag was added to comply with WMF's schema change policy.
It is no longer needed now that the feature is enabled on WMF sites.
External MW installations will run update.php or manually do DB updates
as required before upgrading MW.

Bug: T162517
Change-Id: I5b532d79fd1e8b178490cf2617499ae62967de2c
2017-05-06 20:50:42 +10:00
Aaron Schulz
b89d9b1162 Avoid master queries in loadAndLazyInit() for miser mode
Bug: T92357
Change-Id: Idaef633d9039cdbfa3bd123018f2339c6d489fe4
2017-04-28 22:10:43 -07:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
This, that and the other
1e49fa5d7a Make user_groups queries honor $wgDisableUserGroupExpiry
Follow-up to Icf78cce3f3e362677d10897b8d1103b3df91fa08.

Change-Id: I94102ceb00398940965280d8742a2747631dcc87
2017-01-31 01:21:47 +11:00
This, that and the other
ac8f144c04 Disregard expired user_group rows in special page and API DB queries
An essential follow-up to I93c955dc7a970f78e32aa503c01c67da30971d1a.

Bug: T12493
Change-Id: Icf78cce3f3e362677d10897b8d1103b3df91fa08
2017-01-27 17:05:31 +00:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
16266edff3 Change "slave" => "replica DB" in /includes
Change-Id: Icb716219c9335ff8fa447b1733d04b71d9712bf9
2016-09-05 21:01:01 +00:00
Tim Starling
18938b1ce8 Clear SiteStats process cache after DB update
This fixes a phpunit test error, wherein {{NUMBEROFFILES}} would give
the number of files in the host wiki, not in the temporary database,
when Scribunto was installed, due to a Scribunto phpunit data provider
calling SiteStats::pages().

Change-Id: Ic0d021a72addaa2a13a6b94fd34dccc423de3a8f
2016-08-25 11:44:10 +10:00
Kunal Mehta
f8c7920366 SiteStats: Fix some phpdoc
...and don't pass false as $conds, use an empty array.

Change-Id: I58127f0e7c4edad9345b924a019d64e5d2d6ddd9
2016-08-09 15:10:49 -07:00
Aaron Schulz
ea1f4e8a53 Add TTL_PROC_* constants for clarity
This makes the reason for the cache times more obvious

Change-Id: Ie75df6be77c513feeb6cff3417ccbc124a6a1141
2016-05-06 23:45:41 +00:00
addshore
4561a0eed2 Catch JobQueue errors in SiteStats::jobs
Bug: T133045
Change-Id: I68a9334a63c04609227f4c4e9ae510633543c3d5
2016-04-20 11:07:40 +01:00
addshore
2873ccba94 Remove unused wfGetDB() in SiteStats::jobs
Change-Id: Id818a71a75e4d4cb8397df5b1ff3dfdd3b155ea7
2016-04-19 17:07:50 +01:00
addshore
8a0ae85e0f Remove unused private SiteStats::$groupMemberCounts
Change-Id: I7839a644bca2829098e5ce5d844db667b6d16b26
2016-04-03 08:38:04 +00:00
Sébastien Santoro
1d34f78d94 Allow SiteStatsInit to run on vslow hosts
SiteStatsInits queries could be long running, and so lead to buffers
and undo table issues.

This change allows SiteStatsInits to connect to vslow host.

Bug: T131475
Change-Id: I63c643ef3cc12f38c24374592c55364ba7569a26
2016-04-01 11:35:40 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Timo Tijhof
e8275758fe objectcache: Introduce IExpiringStore for convenient TTL constants
Also consistently use self:: instead of BagOStuff:: for constants
referenced within the BagOStuff class.

Change-Id: I20fde9fa5cddcc9e92fa6a02b05dc7effa846742
2015-10-28 04:07:25 +00:00
Aaron Schulz
9f04548ea9 Fix bad call to Database::getCacheSetOptions()
Change-Id: I8ac756851b4cc7d36129fbd4e01282a151dc2694
2015-10-19 12:57:34 -07:00
Aaron Schulz
4f781c7bf9 Convert some users to WANObjectCache for consistency
These callers don't need to do purges, but can still perfectly
take advantage of this instance over a plain BagOStuff. Namely:
* Replication and snapshot lag awareness
* Preemptive regeneration
* Easy process cache support

The idea is for there to only be one caching class/factory
to use, instead of having rules for picking which one to use.

Change-Id: I8e362df451c0c28731fc853c044c4c4b8e097f01
2015-10-19 16:46:52 +00:00
Aaron Schulz
7c07943eab Update more docs and type hints to use IDatabase
Change-Id: I8c8d85b32a8aba21e14d2a2dde4c25eb509186c1
2015-10-06 18:49:52 -07:00
Aaron Schulz
0f7893f877 Update various @params from DatabaseBase to IDatabase
Change-Id: I98e44cdffb0fc0d729f69f702799139afb988c20
2015-10-05 05:24:29 +00:00
Aaron Schulz
ff88ba3c5f Prevent SiteStats::loadAndLazyInit from reloading in $wgMiserMode
* Large sites can use a chron to handle accumulated drift

Bug: T95426
Change-Id: I59603a7aa329a10f8a7e4d11f3496e8f8243ca42
2015-04-08 09:35:56 -07:00
Chad Horohoe
aa21e125a3 Remove obvious function-level profiling
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.

Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.

Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
2015-01-07 11:14:24 -08:00
Chad Horohoe
90d90dad6e Remove hitcounters and associated code
The hitcounter implementation in MediaWiki is flawed
and needs removal. For proper metrics, it is suggested to use
something like Piwik or Google Analytics.

RFC: https://www.mediawiki.org/wiki/Requests_for_comment/Removing_hit_counters_from_MediaWiki_core
Change-Id: I0e5006a7e8a09c800f8fa4effa9399e8afdd7a57
2014-10-20 13:01:55 -07:00
umherirrender
1c68a1ee86 Cleanup some docs (includes/*.php)
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I783e4dbfe5f6f98b32b9a03ccf6439e13e132bcc
2014-07-24 19:42:24 +02:00
This, that and the other
babe76221c Fix initSiteStats.php maintenance script
It appears not to have worked for some time, as it either calls a protected
function ($counter->refresh()) or an undefined function
($counter->update()) depending on the parameters specified.

Bug: 65214
Change-Id: Ia7d867792b84c98714ec6dbbfef09745e875c8bc
2014-06-29 00:12:16 +00:00
Siebrand Mazeland
c0c39640e4 Make phpcs-strict pass on includes/ (5/~10)
Change-Id: I259f3f11cfc22f3ed1693f9ebd5bd80491b8a6e8
2014-05-11 19:35:32 +00:00
umherirrender
a3983418d5 Fixed some @params documentation (includes/*)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I0056b4a8df243cfc0c5f25378de48f7a35170aca
2014-04-22 13:07:02 +02:00
umherirrender
b058096a58 Remove another ss_active_users check from SiteStats::isSane
Also add a comment to method doc to make sure, nobody will add it again.

Follow-Up: Id8f1cb7035ae0058fb34a6c7bdf7bf9f6c740a69
Follow-Up: I268599be96106e1175fdf9750a2adc9468ebc93c
Change-Id: Ifd6da554b531720aace4497f7fbc2f58c12edc07
2014-02-24 18:17:35 +01:00
Marius Hoch
d8c0373371 Make SiteStats (re)initializing more sane
Don't set ss_active_users back to its default in SiteStatsInit::refresh.
Also remove SiteStatsInit::update and make refresh() protected.
Also don't consider ss_active_users in SiteStats::isSane as that value
isn't going to be fixed by the following SiteStatsInit::refresh call.

Change-Id: I268599be96106e1175fdf9750a2adc9468ebc93c
2014-02-24 00:40:51 +00:00
Tim Starling
64d0dc06a0 Move the DeferredUpdates hierarchy to its own directory
The grouping makes at least as much sense as job/, and certainly makes
more sense than cache/. With directories named after base classes, it is
fairly easy to tell what should go where. The grouping of
DeferredUpdates, DataUpdate and CallableUpdate would surely be
uncontroversial.

The move of SearchUpdate out of search/ demonstrates the conflict between
arrangement by module versus arrangement by type, which is the most
difficult design question here. I think arrangement by type is more
consistent with e.g. the arrangement of the core root, i.e. tests/,
resources/, maintenance/, etc. where a given feature will have its files
split up into a mostly type-based hierarchy.

I also tidied up AutoLoader.php by moving includes/content to the correct
location, sorted alphabetically by subdirectory.

Verified with AutoLoaderTest.

Change-Id: Ib369411d0caca38e72978084aa57348f1b892ed0
2013-11-04 10:32:40 +11:00
Aaron Schulz
167d7149e4 jobqueue: improved performance of JobQueueGroup::getQueuesWithJobs()
* Also added a JobQueueGroup::getQueueSizes() function.
  This function is now used by the SiteStats to make is
  meaningful for when queues are not in the DB. This number
  is already exposed via the API.

bug: 45072
bug: 50635
bug: 9518
Change-Id: I75c16ffa14c963e7f8fb7cb390e6cc4cde0a5804
2013-10-01 03:18:48 +00:00
Aaron Schulz
45daade27c Avoid "Transaction already in progress" errors in SiteStatsUpdate::doUpdate
* Also fixed TRX callback code to match documentation (wrong since 3b7c4f6)

Change-Id: I8c4ea7412e9f96912939c441c68090eacc42b3d4
2013-07-20 12:09:12 -07:00
umherirrender
2f8a5b34e6 Add ss_active_users in SiteStats::isSane
Also changed the array to the full row name, avoids unnecessary string
concat

Change-Id: Id8f1cb7035ae0058fb34a6c7bdf7bf9f6c740a69
2013-05-23 19:05:13 +02:00
Timo Tijhof
4bd5471ca3 docs: Remove odd colons after @todo
Most were this way already:
https://doc.wikimedia.org/mediawiki-core/master/php/html/todo.html

Ran a find/replace on the odd ones. Also made them all
lower case.

Change-Id: I70c6a69344ddebc603e9a1c1d87e3cc4f4f4c560
2013-05-15 06:23:40 +00:00