Commit graph

307 commits

Author SHA1 Message Date
jenkins-bot
3c3c28aa6a Merge "profiler: Drop support for Tideways 4.x in ProfilerXhprof" 2022-11-28 06:35:24 +00:00
Kevin Israel
b3a2c88cea profiler: Drop support for Tideways 4.x in ProfilerXhprof
This branch fails to compile against PHP 7.4.3, our lowest supported
version, and will not be updated to support that or any newer version:

https://tideways.com/profiler/blog/releasing-new-tideways-xhprof-extension

In doing so, remove the Xhprof class. At this point, it doesn't do
anything other than calling either of two functions from whichever
extension is installed, which can just be done in ProfilerXhprof.

Change-Id: Iba1c16f47c68b36982f5d2dcda3bdf4dcb83501d
2022-11-28 06:20:45 +00:00
Umherirrender
c6fecd5ab0 tests: Replace assertEmpty with assertSame
assertSame avoids use of loose comparisons and
allows to check the expected type

Change-Id: If821d1be2bc1ff9f8b70968a339f33c7a8e8880a
2022-11-25 00:42:53 +01:00
Aaron Schulz
23eab3ec39 rdbms: make SqlitePlatform::tableName() apply double quotes
The RDBMs classes treat table names as application-supplied rather
than user-supplied, since it does not make sense for the user to be
able to blindly pick the schema (likely causing query errors). Table
names are generally non-problematic [a-zA-Z0-9_]+ strings. However,
they still should be escaped as identifiers, since they might have
problematic characters. Fix the SQLite backend to apply escaping
here, similar to MySQL and Postgres.

Fix incorrect use of tableName() in DatabaseSqlite::fieldInfo().
It should work the same as in tableExists().

Bug: T72510
Change-Id: I3713f81e6e1eca84cc51c12c9307d4c5dc737043
2022-11-23 00:17:40 +00:00
jenkins-bot
50b187c5ff Merge "BagOStuff: never use the full cache key as metrics key" 2022-11-21 18:02:27 +00:00
daniel
df9a729e57 BagOStuff: never use the full cache key as metrics key
When the cache key doesn't contain a prefix indentifying the collection,
don't use the entire cache key as the metrics name. Doing so may explode
the number of metrics.

Bug: T323357
Change-Id: Ia7ff3f060a36f848c7a74e39070bd62a932120e7
2022-11-21 14:03:44 +01:00
jenkins-bot
2562d05fa1 Merge "Metrics: move metric implementations to subdirectory" 2022-11-17 19:19:44 +00:00
jenkins-bot
e5ad12f3bc Merge "Metrics: expand Sample instance parameters" 2022-11-17 19:19:17 +00:00
jenkins-bot
47c08b164c Merge "Metrics: rename extension to component" 2022-11-17 19:11:07 +00:00
jenkins-bot
380aef9d96 Merge "rdbms: improve TransactionProfiler::silenceForScope()" 2022-11-17 07:53:29 +00:00
Aaron Schulz
6c3ec6b61f rdbms: improve TransactionProfiler::silenceForScope()
Make it able to silence only specific expectations. This means that
rules like "masterConns = 0" and "writes = 0" can be silenced, while
rules like "readQueryTime" and "writeQueryTime" remain in place.

Also, do not increment counters for silenced rules. Otherwise, it
might trigger again on something harmless after the scope ends.

Cleanup event/expectation terminology/naming.

Use time mocking in TransactionProfilerTest.

Bug: T258125
Change-Id: I6e5da5051668fadc5a6ad1955c2cb01ef3d477b6
2022-11-15 23:46:58 +00:00
Reedy
27ee63f8c9 Remove pre PHP 7.4 serialize()/unserialize()
Depends-On: I3b893152ed278a72b5e74ccf91a3381f319bfcd9
Change-Id: I9f69a9316eaec23103262ec2ffd5b85baaf16f3f
2022-11-10 19:25:01 +00:00
Aaron Schulz
f6fec396c5 objectcache: use WRITE_BACKGROUND in WANObjectCache where applicable
This avoids the latency of waiting for a response.

Change yieldStampedeLock() to use delete() instead of changeTTL() since
the PECL memcached client does not support WRITE_BACKGROUND with TOUCH.

Also, simply relayVolatilePurges() into relayVolatilePurge() and tweak
some related documentation for WANObjectCache

Bug: T302623
Change-Id: I16f011b730f34332aa54a48c579de3c6606dafda
2022-11-09 21:17:03 +00:00
Cole White
784598add8 Metrics: move metric implementations to subdirectory
Bug: T240685
Change-Id: I38e8f9fac12542909364f06e0484cc1355796673
2022-11-09 19:53:32 -01:00
Cole White
dd70f21c68 Metrics: expand Sample instance parameters
* Eliminates use of associative array
 * Renames $labels to $labelValues
 * Renames Sample()->getLabels() to getLabelValues()

Bug: T240685
Change-Id: Ic97afd06a98a874bd006946eb8f82e8e2b457e63
2022-11-09 18:08:18 -01:00
Cole White
4a2935a765 Metrics: rename extension to component
Bug: T240685
Change-Id: Ia5ddfddf81685446ab7da3f0b5c415c8989afb7f
2022-11-09 17:41:06 -01:00
jenkins-bot
59ab57573d Merge "objectcache: add "segmentable" option to WANObjectCache" 2022-11-04 00:32:21 +00:00
Aaron Schulz
b24871a15c objectcache: add "segmentable" option to WANObjectCache
This allows for large string value storage similar to that
of direct BagOStuff callers.

Bug: T309126
Change-Id: I22d2d8bb520ef2ea6ccd28016d34db36fe27c3df
2022-11-04 00:12:44 +00:00
Aaron Schulz
85eb9b3e66 rdbms: simplify getLagFromPtHeartbeat() in DatabaseMysqlBase
Avoid expensive SHOW query for sites that were explicitly configured
to use pt-heartbeat but configured without a row SELECT condition.

Reduce excessive code complexity for configuration edge cases.
Remove getLagDetectionMethod() and just use the field instead.

Change-Id: I0d8592fde65d8a143506c55dccd1972a148bd489
2022-11-01 21:33:05 -07:00
jenkins-bot
3a61bdbd43 Merge "objectcache: split out WANObjectCache::setMainValue() helper method" 2022-11-01 23:55:47 +00:00
Aaron Schulz
a454105b48 objectcache: split out WANObjectCache::setMainValue() helper method
Make set() and getWithSetCallback() wrap setMainValue(), avoiding some
duplicate option array resolution logic.

Track set() calls via statsd, now that it can be distinguished from
getWithSetCallback() invocations.

Change-Id: Ia1e7b244b1db3b76314d718b3aea9e6e708235c0
2022-11-01 15:53:48 -07:00
jenkins-bot
3d54b16d8a Merge "DatabaseSQLTest: remove unused variable" 2022-10-28 09:57:52 +00:00
Tim Starling
8983047203 DatabaseSQLTest: remove unused variable
Change-Id: I3087387bb4038df08f0738a756b0608eff8e3133
2022-10-28 13:17:22 +11:00
Timo Tijhof
1cf6f04768 objectcache: Remove WANObjectCacheReaper feature
Introduced in 2017 with I7f14b9ca2533032 (2e5eb693) but remains
unused at WMF, and disabled by default.

Follows-up I62107789fa (9e49260fc9) which added reap to LinkCache
test cases in 2021.

Change-Id: I0654c29a671467dd6b366f462d1c09b90a273413
2022-10-25 23:09:39 +01:00
jenkins-bot
43830cde86 Merge "Use array_key_first()/array_key_last() in some places" 2022-10-21 21:57:34 +00:00
Tim Starling
0077c5da15 Use short array destructuring instead of list()
Introduced in PHP 7.1. Because it's shorter and looks nice.

I used regex replacement.

Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
2022-10-21 15:33:37 +11:00
Tim Starling
1aeccebaf2 Use array_key_first()/array_key_last() in some places
Introduced in PHP 7.3. I used it to replace reset()/end() followed by
key() where the return value of reset() is not captured and internal
pointer iteration is not locally occuring.

I also used it in a couple of places when reset() is absent but
array_key_first() is semantically desired.

Change-Id: I750d3fa71420cbdca5fb00d82ac5ca40821769d4
2022-10-21 15:01:59 +11:00
jenkins-bot
d2b199c517 Merge "objectcache: Deprecate WANObjectCache::reap() and ::reapCheckKey()" 2022-10-21 00:36:34 +00:00
Timo Tijhof
7a6301e03d objectcache: Deprecate WANObjectCache::reap() and ::reapCheckKey()
Follows-up Ia550ef7fe3 (4d3549ad71) which added reap to NameTableStore
in 2018, which was then (mostly) removed again with I63e61e1ab (2c7e4adcea)
for T198561, except one call was left behind in reloadMap(). I've
changed this to match the existing delete call in acquireId().

To be removed in I0654c29a671467dd6.

Change-Id: I7541c9c6d1ef70d552944aca94cbc4d0f7ec0d5b
2022-10-20 14:56:50 +01:00
Aaron Schulz
c1e1512698 rdbms: improve LoadBalancer connection pool reuse (ii)
Make DBConnRef enforce the DB domain selected during its lifetime
and allow more nested and successive use of the same connection handle
via DBConnRef. This can avoid extra connections in some cases where
getConnection()/getConnectionRef() is used.

Also:
* Reduce the number of connection pools arrays from six to two
* Merge getLocalConnection()/getForeignConnection() into one method
* Expand various related code comments

Since LoadBalancer::getReadOnlyReason() no longer user the local domain
but rather DOMAIN_ANY, it should not result in "USE" errors if the local
domain does not have a database on the server.

This version of the patch removes the unused reuseConnectionInternal()
method (the method was previously added back to the patch by mistake).

Bug: T226595
Change-Id: I62502f4de4f86a54f25be1699c4d1a1c1baee60b
2022-10-18 17:23:44 -07:00
jenkins-bot
6198dc4d00 Merge "Add some helpers to filter properties out of var_dump()" 2022-10-12 16:59:43 +00:00
Tim Starling
94e20db64b Add some helpers to filter properties out of var_dump()
Dumping many kinds of object with var_dump() leads to dumping the whole
address space, because a chain of references leads to MediaWikiServices.

So, introduce a trait which replaces the problematic properties with a
placeholder, if their doc comment contains @noVarDump.

Bug: T277618
Change-Id: Ifa685c26fbc5317d0359544289ec3f433ec4fedf
2022-10-12 12:42:23 -04:00
Amir Sarabadani
fdc246801e rdbms: Introduce DatabaseFlagsHolder and move some internal logic there
Bug: T299691
Change-Id: If9da21bebf1b233da38ed198f35d3dcd51c356a5
2022-10-11 17:59:47 +02:00
Daimona Eaytoy
7a494d1fa4 tests: Upgrade PHPUnit from 8.5+ to 9.5+
* DeprecatedHooksTest: Don't use assertContains().
* Replace uses of deprecated asserts:
  - assertFileNotExists() -> assertFileDoesNotExist()
* Update hierarchy of MediaWikiPHPUnitResultPrinter, since ResultPrinter
  is an interface in PHPUnit 9.
* Remove temporary forward-compat methods.
* Remove directories that don't exist from tests/phpunit/suite.xml, since
  they now make PHPUnit exit:
   - tests/phpunit/skins, it used to have SideBarTest, then moved to
     tests/phpunit/includes/skins
   - tests/phpunit/documentation, it used to have ReleaseNotesTest, then
     moved to tests/phpunit/unit/documentation
* Update configuration with --migrate-configuration and reformat.
* Avoid redefining getMockBuilder() in
  ActionModuleBasedHandlerTestTrait, use a @method annotation instead.
* In RCCacheEntryFactoryTest, avoid using internal PHPUnit logic for
  HTML validation, and use native PHP methods instead. The code was
  copied from Xml::load (moved to \Xml\Loader::load in PHPUnit 9) and
  simplified for this use case.

Bug: T243600
Bug: T262076
Change-Id: I851b9158b73d0cfc315eed9d63b15c54b05895e3
2022-10-08 02:03:55 +02:00
Daimona Eaytoy
ee672592d8 tests: Replace assertRegExp with assertMatchesRegularExpression
And also assertNotRegExp -> assertDoesNotMatchRegularExpression. The
methods were renamed in PHPUnit 9.

Done automatically with:
  grep -rl assertRegExp tests/ | xargs sed -r -i "s/>assertRegExp\(/>assertMatchesRegularExpression\(/"
  grep -rl assertNotRegExp tests/ | xargs sed -r -i "s/>assertNotRegExp\(/>assertDoesNotMatchRegularExpression\(/"

Split out from Ifdba0f9e98eb6bce4590b7eb73170c51a697d7c6 so that it
remains smaller and easier to review.

Also make a test use MediaWikiUnitTestCase (it's already in the unit/
dir) so that it can access the forward-compat method.

Bug: T243600
Change-Id: Ifa279d5f201d7abeebece292141ebface8278046
2022-10-07 14:13:16 -04:00
Alexander Vorwerk
872970a21c Drop some pre-7.4 php compat code
Change-Id: Ie92e4b2722692c64f38557bfcd3687168d245e8e
(cherry picked from commit eec198b245901d9a5dbe52a426e909ffbff32ac0)
2022-09-29 14:40:13 +00:00
Ladsgroup
4edae94817 Revert "rdbms: improve LoadBalancer connection pool reuse"
This reverts commit 76be4dca81.

Reason for revert: Caused T318904

Bug: T318904
Change-Id: I92f23205b4d62f69c0f27af40f972f104d1a7a7c
2022-09-29 12:12:14 +00:00
Aaron Schulz
76be4dca81 rdbms: improve LoadBalancer connection pool reuse
Make DBConnRef enforce the DB domain selected during its lifetime
and allow more nested and successive use of the same connection handle
via DBConnRef. This can avoid extra connections in some cases where
getConnection()/getConnectionRef() is used.

Also:
* Reduce the number of connection pools arrays from six to two
* Merge getLocalConnection()/getForeignConnection() into one method
* Expand various related code comments

Bug: T226595
Depends-On: If808cbab429d41e1f2289683533e4a781a4bdf5e
Change-Id: I540b08920997c57cad6445ddb09d8e663eaf4714
2022-09-23 10:53:58 +10:00
jenkins-bot
89c4e47745 Merge "rdbms: Allow SubQuery objects in SelectQueryBuilder as table" 2022-09-14 20:49:54 +00:00
jenkins-bot
ca91baed16 Merge "SQLPlatform: Introduce buildComparison()" 2022-09-13 10:20:08 +00:00
jenkins-bot
779b83705f Merge "Move unit tests for SQLPlatform to a separate file" 2022-09-13 10:12:52 +00:00
Amir Sarabadani
7e381eb74d rdbms: Allow SubQuery objects in SelectQueryBuilder as table
That makes everything simpler, it already does set the table as subquery
when the table passed as SelectQueryBuilder.

Bug: T314189
Change-Id: Id9d6c1d0d4c40f43ab0d2df39766b77ed06b9a76
2022-09-12 09:56:31 +02:00
Bartosz Dziewoński
ec79aa3943 SQLPlatform: Introduce buildComparison()
Builds a condition comparing multiple values, for use with indexes
that cover multiple fields, common when e.g. paging through results
or doing batch operations. Can also be to generate a simple comparison
without writing raw SQL (see T210206).

Update a few manually constructed conditions to use this method.
There are more maintenance scripts and API classes that use the
same patterns, but this is a start.

As you can see by the code I'm replacing, there are many ways to do
this. I picked the one used by maintenance/TableCleanup.php, since
I found it the easiest to understand.

Change-Id: Ic368a87fb5ce4c13608b03206cd68518ec9732d4
2022-09-10 04:22:19 +02:00
Bartosz Dziewoński
af60bf699c Move unit tests for SQLPlatform to a separate file
Change-Id: I91aec0898d2ae7b348c423f6f31b1596f7f42d91
2022-09-10 04:22:18 +02:00
Timo Tijhof
6f4db32e0c rdbms: Remove unused channelsMatch() from internal MySQLPrimaryPos
Remove it from the interface as well. The interface is marked stable
to implement for (theoretical) database extensions.

This is not a breaking change as any existing implementations of the
method may continue to exist, and continue to be uncalled by core.

The interface is not stable to type or call, and the built-in
implementation is internal/unexposed.

Change-Id: Iced3ab3dfca70dec2958eb9d750e7025b95b7de3
2022-09-07 22:08:04 +00:00
jenkins-bot
5a735e81c1 Merge "rdbms: Remove unused getLogName() from internal MySQLPrimaryPos" 2022-09-05 07:09:31 +00:00
Timo Tijhof
b1c07ad652 rdbms: Remove unused getLogName() from internal MySQLPrimaryPos
Also hoist up the `@covers` annotation for the test class overall,
and widen some of the more specific ones to make up for the fact
that this code has recently been refactored and thus is now part
of different classes.

Change-Id: I67d19502fd191042c902d8b48d8cbd3ff76473d6
2022-09-05 02:16:52 +02:00
Alexander Vorwerk
acb5c2f4e0 Update doctrine/dbal versions
Change-Id: I60b66ff86e4c624570ca0514c28a8b7a803cfcc7
2022-09-04 23:41:34 +00:00
daniel
96af469d87 rdbms: Use JSON to store ChronologyProtector position data
PHP serialization is brittle, so serialize DBPrimaryPos objects to JSON
structures before writing them to the stash.

BREAKING CHANGE: This adds to methods to the DBPrimaryPos interface.
There are no known implementations of this interface outside MediaWiki
core. There appears to be no good alternative to adding these methods to
the interface. The breaking change was announced on wikitech-l on August 31.

Bug: T316601
Change-Id: I78096d893a6a2c000c35673b78e36b52369560e5
2022-09-04 15:38:00 +00:00
jenkins-bot
2b6e081ff3 Merge "rdbms: log invalid INDEX clauses with multi-table selects" 2022-09-02 02:06:53 +00:00