Commit graph

67 commits

Author SHA1 Message Date
Bartosz Dziewoński
13289a1269 Use real type hints for services etc. in includes/block/
Mostly used find-and-replace:

Find:
/\*[\*\s]+@var (I?[A-Z](\w+)(?:Interface)?)[\s\*]+/\s*(private|protected|public) (\$[a-z]\w+;\n)((?=\s*/\*[\*\s]+@var (I?[A-Z](\w+)(?:Interface)?))\n|)
Replace with:
\3 \1 \4

More could be done, but to keep this patch reasonably sized, I only
changed the most obvious and unambiguously correct cases.

In some cases, I also removed redundant doc comments on the
constructor, and re-ordered the properties to match the constructor.

Change-Id: I819ed771c915293663856c577a481d607b76ed80
2024-07-31 08:54:31 +00:00
Umherirrender
fc9e42823b rdbms: Create IReadableDatabase::andExpr() / ::orExpr()
Avoid the call to internal constructor of AndExpressionGroup and
OrExpressionGroup by creating a factory function similiar as the
IReadableDatabase::expr function for Expression objects.

This is also a replacement for calls to ISQLPlatform::makeList with
LIST_AND or LIST_OR argument to reduce passing sql as string to the
query builders.

Created two functions to allow the return type to be set for both
expression group to allow further calls of ->and() or ->or() on the
returned object.
Depending on the length of the array argument to makeList() it is
sometimes hard to see if the list gets converted to AND or OR, having
the operator in the function name makes it easier to read, so two
functions are helpful in this case as well.

Bug: T358961
Change-Id: Ica29689cbd0b111b099bb09b20845f85ae4c3376
2024-07-11 15:29:20 +00:00
Umherirrender
a9adc61d35 Use RawSQLValue for some SET clauses in update/upsert
Change-Id: I03c068c4bf098ab073989a0ea32a0f87fe878727
2024-05-29 19:48:09 +00:00
Umherirrender
be157850ad Replace IDatabase::buildLike with expression builder
Bug: T361023
Change-Id: I0fcb61ff1b15931477bce3c8f850a8ce97405a36
2024-05-18 12:29:17 +00:00
Tim Starling
a5372218b7 block: Deprecate and stop using $wgBlockTargetMigrationStage
* Deprecate and stop using $wgBlockTargetMigrationStage. Remove
  block_target migration code.
* Make the $schema parameters to DatabaseBlockStore methods default to
  SCHEMA_BLOCK. Avoid passing these parameters where possible.
* Remove cleanupBlocks.php
* Deprecate DatabaseBlock static methods which try to present the old
  schema for b/c.

Bug: T362133
Change-Id: I845bad8cc09a4528fef46a6f9d490ebdec881d99
2024-05-10 15:54:17 +10:00
Tim Starling
eb391335bb block: Fix overly broad block selection in getRangeCond()
In the new schema, getRangeCond() with a specified range of say
1.2.0.0/16 would match blocks of narrower range with the same start,
e.g. 1.2.0.0/24, because a match of bt_ip_hex was sufficient, but that
is the same as the start of the range.

So, when searching for single IP blocks, add a condition requiring
bt_range_start IS NULL.

This affects the bkip parameter to ApiQueryBlocks and searches from
Special:BlockList.

Add regression test.

Change-Id: Ie3943befb8a02cc4939624f7bad48ae50056cd7d
2024-05-08 09:12:43 +10:00
Umherirrender
d38b9e4cfa Migrate to IReadableDatabase::newSelectQueryBuilder with queryInfo
This is using SelectQueryBuilder::queryInfo to interact with legacy code

In the long term this needs refactor to get a SelectQueryBuilder from
the class instead of a query info array.

Bug: T344971
Depends-On: I5dff80a2819bba4e923654ef31816a54269354b3
Change-Id: Ibf44e7de30e71fe8dd6447f77dfd0bf726be0a9b
2024-05-04 19:56:11 +00:00
Umherirrender
98da56b588 block: Typehint delta in DatabaseBlockStore for sql safety
A non-typehinted argument could be a raw sql string ending in the sql
statement, using typehint makes sure this is an safe integer

Change-Id: I2c1eef4dd2a05b9bb1329b79749502a242804687
2024-04-18 21:41:52 +02:00
jenkins-bot
873346377d Merge "block: Fix DatabaseBlockStore::newLoad index usage" 2024-03-18 23:57:04 +00:00
Tim Starling
652ac0b375 block: Fix DatabaseBlockStore::newLoad index usage
There's no index on bt_user_text alone, only one that is prefixed with
bt_ip_hex. So include a condition on bt_ip_hex when searching for blocks
by username.

Bug: T360165
Change-Id: I99ed17ed66449ab7c3f76c98d90f91a75fbd8fd5
2024-03-18 09:36:33 +11:00
Umherirrender
8f5c33bfe0 Use IConnectionProvider in DatabaseBlockStore
Allows to type-hint IReadableDatabase for the replica

Change-Id: I7b08808c754e9570a78de03bf32deb6c7d561c9c
2024-03-14 00:24:29 +01:00
Tim Starling
dcbabb1a1d Fix autoblock update
When updating an autoblock due to a reblock, use the correct reason
message instead of the parent block reason.

If the reblock causes the parent block to expire before the autoblock,
adjust the autoblock expiry time downwards.

Adapt testUpdateBlock() to be a regression test for these two bugs.

Bug: T351173
Change-Id: I19843e4971106250cf9644ef68d68d6e33c6e3ab
2024-02-28 13:13:47 +11:00
Umherirrender
8df2cbe287 block: Pass wikiId to DatabaseBlock::getId in DatabaseBlockStore
For cross-wiki autoblock support

Bug: T358208
Change-Id: I9396f289237769a7b5fc9e1a8c575116abaf7609
2024-02-22 21:35:23 +01:00
Tim Starling
45b88830d3 block: Include bt_auto in acquireTarget() conditions
When searching for an existing block_target row to use for a new block
of an IP address, include bt_auto in the conditions. Otherwise, if there
is an existing autoblock, the new block will fail due to a mismatch of
bt_count, leaking information about the private IP address.

Bug: T357366
Change-Id: If873ca590aadd29ab0b9d672a99438d70c4292d9
2024-02-13 13:20:38 +11:00
Tim Starling
3cdd4d6256 block: When inserting in write-both mode, ensure the ID remains synchronised
The block ID is exposed in various places and must be consistent between
the two schemas.

Bug: T357366
Change-Id: I5b104242341ebcfec8f33a904c771d61d236dca2
2024-02-13 13:20:36 +11:00
Alexander Vorwerk
e838ef9773 Don't allow passing a db to DatabaseBlockStore::insertBlock
Bug: T291849
Change-Id: I5f5cec791eb2999d7295440248f74fba79156b6f
2024-01-01 14:23:24 +01:00
Alexander Vorwerk
fadc67b66b DatabaseBlockStore: Fix type of $expectedTargetCount in doc
can be null

Change-Id: I0ead62a935c9b773055eec609305d02e7edb94c3
2024-01-01 12:56:17 +00:00
Dreamy Jazz
cda474bc33 Use username for lookup for non-existing user as the vague target
Why:
* When a vague target that is a non-existing user, the code in
  DatabaseBlockStore should lookup by the username to be consistent
  with the previous behaviour.

What:
* In DatabaseBlockStore::newLoad, if the vague target type is a
  non existent user, then lookup by the username.

Change-Id: I1d982f12fc4be0dde5a26228ed407d4612f2b187
2023-12-21 13:24:28 +00:00
Tim Starling
3c04561049 Remove weird special case from BlockUtils::parseBlockTarget
* If a caller asks for a block and gives a string target which can't be
  a valid block, then there are no blocks for that string. It seems to
  do more harm than good to throw an exception since the string can come
  from the user. This is an alternative fix for T31797.
* In Article::getRobotPolicy(), noindex subpages of blocked user pages,
  as was requested in T13443.

Bug: T13443
Bug: T31797
Change-Id: I54c453ab202ffcfa8ebc81308de6417614c6e7e8
2023-11-29 22:25:18 -05:00
Tim Starling
d6727856fd Support new block schema
Support migration stages when reading and writing blocks.

I tried to set it up for an easy next stage, in which support for the
old schema is removed. I tried to avoid factoring out of shared code
between the two schemas, so that the old schema cases can simply be
deleted without the need to revert unnecessary abstractions.

However, I added HideUserUtils to factor out ipb_deleted queries. Code
review showed that this was already quite complex, with multiple
approaches to the problem, so it benefits from refactoring even without
the schema abstraction.

HideUserUtils is a service rather than a standalone class to support
unit tests, since unit tests do not allow global config access. When
the migration stage config is removed, it will be a service with no
constructor parameters -- an unnecessary abstraction which should
ideally be resolved at that time.

When interpreting result rows, it is possible to share code by using
field aliases. But when constructing WHERE conditions, the actual field
names need to be used, so the migration is more intrusive in
ApiQueryBlocks and SpecialBlockList, where complex conditions are used.

Bug: T346293
Bug: T51504
Bug: T349883
Change-Id: I408acf7a57b0100fe18c455fc13141277a598925
2023-11-29 13:31:42 +11:00
James D. Forrester
67217d08df Namespace remaining files under includes/deferred
Bug: T166010
Change-Id: Ibd40734b96fd2900e3ce12239d09becfb4150059
2023-11-22 10:08:53 -05:00
Tim Starling
d06ac1a3cd Move DatabaseBlock read query methods to DatabaseBlockStore
* Move to DatabaseBlockStore the DatabaseBlock methods newFromID,
  getQueryInfo, getRangeCond, newFromRow, isExemptedFromAutoblocks,
  doAutoblock, updateTimestamp, getAutoblockExpiry, newFromTarget,
  newListFromTarget.
* Split DatabaseBlock::getBlocksForIPList. Now
  BlockManager::getBlocksForIPList() is responsible for XFF header
  validation and trusted proxy handling. DatabaseBlockStore::
  newListFromIPs() just does the queries and constructs the Block
  objects.
* In DatabaseBlockStore::newFromRow() and doAutoblock(), use the
  DatabaseBlock constructor instead of calling many setter methods. Add
  constructor options decodedExpiry, decodedTimestamp, id,
  parentBlockId and restrictions to support this.
* Move isExemptedFromAutoblocks() to its own service. Remove the cache
  since in my testing with production eval.php, the WAN cache fetch is
  10 times slower than just using the message cache, contradicting the
  comment written in 2008.
* Fix AuthManagerTest which was previously passing an unrecognised
  "restrictions" option to DatabaseBlock. Now that the option actually
  works, we have to use the right type.

Bug: T255433
Change-Id: I5049e60be1681f67fcca133e569e315792dc42dd
2023-11-03 11:00:57 +11:00
Tim Starling
6629bfb13f DatabaseBlockStore: factor out calls to LoadBalancer
And replace deprecated getConnectionRef() with getConnection().

Change-Id: I2b78c0376f69cba19e7106c046a00a289d45aad6
2023-11-01 13:13:10 +11:00
Bartosz Dziewoński
978d739bc6 Replace single-value $db->buildComparison() with $db->expr()
Find:
->buildComparison\( ('..?'), \[(\s*)([^\],]+) => ([^\],]+)(\s*)\] \)

Replace with:
->expr($2$3, $1, $4$5)

Change-Id: I2cfc3070c2a08fc3888ad48a995f7d79198cc336
2023-10-22 01:05:47 +02:00
jenkins-bot
e291c4044e Merge "rdbms: Add support for per-domain calls in ReadOnlyMode" 2023-09-10 03:38:17 +00:00
Amir Sarabadani
4fccf6bf30 rdbms: Add support for per-domain calls in ReadOnlyMode
That would remove the need for passing around LB.

ReadOnlyMode code needs a lot of clean up. In follow up patches, I want
to inject WAN and possibly APCu to this and remove that from LB itself
to decouple LB/LBF from BagOStuff.

In even later patches, I want LB/LBF to set the ReadOnly in the service
instead and reverse the coupling.

Bug: T343917
Change-Id: Ibe7f80292d911fc8953e59f74264c499ab1700d0
2023-09-10 03:23:13 +00:00
Daimona Eaytoy
3f35734ac0 Remove redundant empty() constructs
empty() only makes sense when the expression it checks is possibly
undefined, otherwise it's equivalent to a truthiness check with the
additional downside of suppressing errors when it's not wanted.

Replace it with simple truthiness checks, using strict comparison when
that seems to help with polymorphic variables.

These were caught by a bespoke phan plugin.

Change-Id: I52999e5286a0d9ad70b0da40b41b9f998ecb990e
2023-09-08 23:18:11 +02:00
Tim Starling
95bd40b25c In query builders, use insertInto() and deleteFrom() instead of insert() and delete()
The design principle for SelectQueryBuilder was to make the chained
builder calls look as much like SQL as possible, so that developers
could leverage their knowledge of SQL to understand what the query
builder is doing.

That's why SelectQueryBuilder::select() takes a list of fields, and by
the same principle, it makes sense for UpdateQueryBuilder::update() to
take a table. However with "insert" and "delete", the SQL designers
chose to add prepositions "into" and "from", and I think it makes sense
to follow that here.

In terms of natural language, we update a table, but we don't delete a
table, or insert a table. We delete rows from a table, or insert rows
into a table. The table is not the object of the verb.

So, add insertInto() as an alias for insert(), and add deleteFrom() as
an alias for delete(). Use the new methods in MW core callers where
PHPStorm knows the type.

Change-Id: Idb327a54a57a0fb2288ea067472c1e9727016000
2023-09-08 10:16:08 +10:00
Tim Starling
1ee50f1a77 Blocks documentation review
* Use a consistent single-line comment style.
* Start each sentence with a capital letter.
* Use imperative mood.
* Put a space between the adjective and the noun in "hardblock" and
  "rangeblock".
* Capitalise acronyms.
* Break long lines.
* Remove excessively obvious or off-topic comments.
* Indent the non-initial lines of a multi-line @ command.
* Don't try to apply @deprecated to a parameter. Doxygen and PHPStorm
  interpret this as method deprecation.

Bug: T345683
Change-Id: I7df126f7a031dde241dd46f66e5e83722f9b383c
2023-09-06 13:48:07 +10:00
Amir Sarabadani
98d6503f65 Mass migrate Database::insert calls to InsertQueryBuilder
Done automatically based on a php parser written on top of ANTLR4

Bug: T335377
Change-Id: Ie8fabc594edab18e55cb1d5bbf573762106e3d71
2023-08-07 19:44:27 +02:00
jenkins-bot
25bcb59653 Merge "block: Simplify deletion by parent id for BlockRestrictionStore" 2023-08-02 16:55:20 +00:00
Amir Sarabadani
7432b21816 Migrate more calls of Database::select* to SelectQueryBuilder
Using a php parser written on top of ANTLR4, done semi-automatically.

I checked everything and made adjustments.

Bug: T311866
Change-Id: I6150c6909bce8f3dbd745a26380cc0af9d9c547f
2023-07-26 13:01:28 +02:00
Umherirrender
e11a6165ec block: Simplify deletion by parent id for BlockRestrictionStore
Remove BlockRestrictionStore::deleteByBlockId and replace by
select on parent id and delete on primary key.
This avoids that restriction store needs to determine the rows via
deleteJoin and the block store via parent select. Just use the primary
key in both functions for deletion and combine the delete for parent and
normal block id where possible.
In case there are no parent blocks this also removes a possible gap
lock. In case the unblock and the autoblock happens in same second the
autoblock may there without its block

Change-Id: I274d35834ce1e3e1d67fabd698d9a1cb3de9687a
2023-07-13 00:17:37 +00:00
Umherirrender
fe40101587 Replace IDatabase::delete with DeleteQueryBuilder
Change-Id: Ie0c1c955ca1a7028f75f24563fdeb9f94285af30
2023-06-21 17:50:31 +02:00
Umherirrender
a274949536 block: Cast block ids to integer on database queries
Change-Id: Ie71e31b985013b6a7e2d6e38f3ba322e30ffc5d9
2023-06-14 22:11:09 +00:00
Daimona Eaytoy
4d5cd3a84f Replace deprecated MWException
Bug: T328220
Change-Id: I66be7a6dd752d6b9c254beb65f4eb5ace3c89776
2023-06-09 17:21:12 +02:00
Amir Sarabadani
b55c501fe0 Migrate Database::update() to UpdateQueryBuilder
I did this using a script written on top of antlr4 parser so it doesn't
have some clean ups a human would do but it's pretty nice already.

Bug: T330640
Change-Id: I608566700c6d737ee986bf47dda87effc69614d6
2023-06-08 12:34:20 +02:00
Aaron Schulz
ac1819cc32 Remove uses Database::update()/Database::insert() return values
Change-Id: I527ea0900cc76faac9506b2a43a2b1522b35dfbd
2023-05-27 02:19:55 +00:00
Alexander Vorwerk
31d063ca71 Deprecate passing a db to DatabaseBlockStore::insertBlock()
Instead of passing a custom database to DatabaseBlockStore::instertBlock
callers should rather just fetch a correct DatabaseBlockStore
using the new DatabaseBlockStoreFactory service.

Bug: T291849
Change-Id: Ia1319f35bbc5bd786d1d96d330f94255d7b3582a
2023-05-20 11:02:45 +02:00
jenkins-bot
a697d50bcb Merge "rdbms: Move ReadOnlyMode and ConfiguredReadOnlyMode to rdbms library" 2023-05-05 00:15:27 +00:00
Amir Sarabadani
623d3dc419 rdbms: Move ReadOnlyMode and ConfiguredReadOnlyMode to rdbms library
The db/ directory does not have an owner and it's a mess in general.
These classes don't depend on anything in core except the rdbms library.
Let's simply move it there. In other words, Krinkle made me do it.

Since the class was moved in I6202e52ba73 merged less than a week ago,
no need to alias anything.

Bug: T321882
Change-Id: I24ceeb8bf765a50f441270136acd612359d50aa2
2023-05-04 23:41:21 +02:00
Aaron Schulz
56b74086b8 block: always check affectedRows() in DatabaseBlockStore::insertBlock()
The code uses "IGNORE", which suggests that it recognises the race
condition where the new row is created by another web request in the
mean time.

When that happens, the remaining lines probably shouldn't run.

Change-Id: Ic69d5153bdfba5211e9474e15d53a22c0f591085
2023-05-04 18:23:03 +00:00
Amir Sarabadani
d59ffa4202 Reorg: Move ReadOnlyMode classes to db/
They are not suitable to go to rdbms library as they depend on mediawiki
pieces, the second best place is the db/ directory.

Bug: T321882
Change-Id: I6202e52ba7306d74261206c2ba7930c5f1a0a18e
2023-04-28 12:13:01 +02:00
Alexander Vorwerk
ef52ed41bb Introduce a DatabaseBlockStoreFactory
The DatabaseBlockStoreFactory is a new service which allows fetching
of a wiki-specific DatabaseBlockStore by fetching the correct
LoadBalancer via a LBFactory and then injecting it into the
DatabaseBlockStore. It also creates a wiki-specific ReadOnlyMode
and BlockRestrictionStore, which are also being injected.

This is being done as a part of the project of supporting cross-wiki
blocking.

The future plan is that BlockUser and UnblockUser are being updated to
properly support creating, updating and deleting cross-wiki blocks.
BlockUser and UnblockUser are then going to use a newly injected
DatabaseBlockStoreFactory to fetch the correct DatabaseBlockStore to
then insert/update/remove the block into/from the correct database.

Bug: T291849
Change-Id: I1051beedfd67c4e2c546b9eec04b6d592d969af3
2023-02-13 01:59:55 +00:00
Alexander Vorwerk
f6bd18d6c2 Split a base class out of CommentStore
so that extensions (i.e. CheckUser) can implement their own comment
store without having a lot of code duplication

basically the comment store version of I3a6486532f2ef36

Bug: T233004
Change-Id: Ib40f99e00a514d41776ce521baf113e46d37e9cd
2023-01-01 22:34:36 +00:00
Aryeh Gregor
7b791474a5 Use MainConfigNames instead of string literals, #4
Now largely automated:

VARS=$(grep -o "'[A-Za-z0-9_]*'" includes/MainConfigNames.php | \
  tr "\n" '|' | sed "s/|$/\n/;s/'//g")
sed -i -E "s/'($VARS)'/MainConfigNames::\1/g" \
  $(grep -ERIl "'($VARS)'" includes/)

Then git add -p with lots of error-prone manual checking. Then
semi-manually add all the necessary "use" lines:

vim $(grep -L 'use MediaWiki\\MainConfigNames;' \
  $(git diff --cached --name-only --diff-filter=M HEAD^))

I didn't bother fixing lines that were over 100 characters unless they
were over 120 and triggered phpcs.

Bug: T305805
Change-Id: I74e0ab511abecb276717ad4276a124760a268147
2022-04-26 19:03:37 +03:00
Timo Tijhof
fb7e47fd5a block: Use $wgUpdateRowsPerQuery for purgeExpiredBlocks()
Follows-up I052735edbfac.

Bug: T301742
Change-Id: Iec989945f4c592d7569e62199dd8190c535acae7
2022-03-04 22:40:53 +00:00
jenkins-bot
e876068ee9 Merge "block: Add a limit to DatabaseBlockStore::purgeExpiredBlocks" 2022-02-28 08:42:33 +00:00
Amir Sarabadani
d344b90f59 block: Add a limit to DatabaseBlockStore::purgeExpiredBlocks
Making it run without limit can easily cause a read-only by creating a
massive transaction choking the replication. It's fine if the values
stay there until the next couple of blocks.

Bug: T301742
Change-Id: I052735edbfaca8821bf6dcbf62737890d3855822
2022-02-28 09:21:47 +01:00
Umherirrender
ba2c15abb2 block: Avoid use of null block id and blocker
Use a local variable to allow phan to track the not-null part

Found by phan strict checks

Change-Id: I32e023d3dbd70e987c79527a6936209d62deffcf
2022-02-26 10:35:10 +01:00