Commit graph

497 commits

Author SHA1 Message Date
Daimona Eaytoy
535d7abf59 phpunit: Mass-replace setMethods with onlyMethods and adjust
Ended up using
  grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'

special-casing setMethods( null ) -> onlyMethods( [] )

and then manual fix of failing test (from PS2 onwards).

Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
2021-04-16 20:15:00 +02:00
jenkins-bot
37ab31298c Merge "Remove $actor field from UsererIdentityValue" 2021-04-14 14:40:12 +00:00
daniel
fed7f0b179 Remove $actor field from UsererIdentityValue
Code that needs to store an actor ID in the database to
represent a UserIdentity, or needs to construct a UserIdentity based on
an actor ID loaded from the database, should use the ActorNormalization
service.

Note: The getActorId() method is removed from the UserIdentity interface,
but all concrete classes continue to support it for now.
UsererIdentityValue::getActorId() is hard deprecated and should
be removed in 1.37. It always returns 0.
User::getActorId() is not deprecated at this point.

Bug: T274179
Depends-On: Id2b3ddf6a2a7cdf90f8936a69148d2cce6fde237
Change-Id: I9925906d11e47efaec3c1f48d5cb3f9896a982c1
2021-04-13 18:18:06 +00:00
Petr Pchelko
d1f481f242 ParserCache: only use in-process caching for metadata
CachedBagOStuff caches negatives, so it breaks PoolCounter.
We only need to cache metadata in-process, since it's commonly
used twice within the request.

Bug: T277829
Change-Id: I11a147c24b6cdb275b521b48802d6f3d0e1a4387
2021-04-06 17:53:38 -06:00
Petr Pchelko
f642215aed Convert ParserCache to PageRecord
ParserOptions not updated cause they depend on Title::getLanguage
implementation.

Tests converted to not require a DB anymore. Can't be proper unit
tests yet due to globals in ParserOptions and fake time hacks,
but exec time does go down from 70 seconds to 9 seconds.

Page content model is still emitted in the metrics since
it was considered useful. Should be removed when we get
something like a page type concept.

Change-Id: Ib16fd0b5b87ffc3cb4d21f4aa43d1203cb7206d2
2021-04-02 21:14:54 -06:00
James D. Forrester
5c83c8952c tests: Avoid problematic language in comments and internal code
We can't avoid this entirely where we're testing upstream features
that are yet to be renamed, but this is a start.

Bug:T277986
Change-Id: I5c1ae717cc008ca147ea31fe35c44b19d1fe5d98
2021-03-28 21:21:28 -07:00
Umherirrender
d01d47683c Fix spacing after yield and use statements
Change-Id: Iacb93e96168ec0cd895130c5c8f66b6b44317e34
2021-03-26 23:55:58 +01:00
Petr Pchelko
7bf51ccef3 Convert ParserOptions to UserIdentity
We still need a lot of refactoring in ParserOptions
constructions, but for now converting the public interface
should be enough.

Change-Id: I04663c39ca037129b827b33555c3f59def5f9b59
2021-03-24 09:40:42 -06:00
DannyS712
24d26873e3 CoreParserFunctionsTest: fix duplication
testSubjectpagename was probably copied from
testTalkpagename and so uses the wrong data provider
and tests the wrong code

Change-Id: Ied4023a8fdd57a1c2b538eca6034c37e94967bc7
2021-03-22 01:48:03 +00:00
Petr Pchelko
0f0a11f6dc Make Parser use UserIdentity instead of User
Change-Id: Idf8578e88af1fd4824f49417a200b16befdbca51
2021-03-17 13:51:52 -06:00
jenkins-bot
76a2acf48a Merge "Parser: initialize preprocessor in constructor" 2021-03-17 16:17:30 +00:00
daniel
afa236ef53 RevisionStore: getRevisionbyId should take a PageIdentity
Providing a PageIdentity to getRevisionById removes the need
to look up page related data, and to construct a Title.

Bug: T275531
Bug: T273284
Depends-On: I8d0dc30ecd0c0c65c337170d271c2e2531914686
Change-Id: I683362ccd669acf2aadf56853afeb1ca6dee6f8c
2021-03-17 13:46:41 +00:00
C. Scott Ananian
4497f99796 Parser: initialize preprocessor in constructor
Initializing the preprocessor in the constructor allows better
dependency injection, and removes code complexity caused by
lazy initialization.  Any use of the parser is going to end
up creating the preprocessor in any case, so deferring the
initialization doesn't save any performance.  (Best performance
is given by not creating the Parser in the first place if it
is not needed, which is what DI allows.)

Old code tried to unbreak cyclic dependencies by setting the
preprocessor to null.  This is somewhat of a lost cause,
since there are a number of other cyclic dependencies
involving the parser, including StripState, LinkHolders,
etc.  The code complexity is not worth it, given how
ineffective it is in any case.

This is part of T275160 in so far as it allows
Parser::getPreprocessor() to be a simple getter, and thus
(once this patch is merged) we can safely replace any
direct access to Parser::$mPreprocessor with a call to
Parser::getPreprocessor().

Bug: T275160
Change-Id: I38c6fe7d5a97badffdbf34d8b9d725756ed86514
2021-03-16 22:37:40 +00:00
C. Scott Ananian
115410f077 Parser::__construct(): Remove deprecated argument variants
A number of different argument variants were deprecated in 1.34,
and direct calls to the Parser constructor were deprecated at the
same time (a ParserFactory should be used instead).  These were
hard-deprecated in 1.35.  They should be safe to remove now.

Code search:
https://codesearch.wmcloud.org/deployed/?q=new%20Parser%5C%28&i=nope&files=%5C.php%24&excludeFiles=&repos=

Bug: T236811
Change-Id: I58f7b3ba1b1d62851b2db71197a8d9129e8d473d
2021-03-16 19:41:45 +00:00
C. Scott Ananian
1fd4a7af4e Introduce Tidy service
Refactor the old MWTidy singleton as a DI service.

Change-Id: I95605ea5fd22f53a7f90fe07a6a73fa6c959597a
2021-03-15 17:22:36 -04:00
Tim Starling
f28578f9da Make ParserOptionsTest work on PHP<7.4.9 with opcache enabled
Instead of refusing to run MediaWiki at all, just fix the test which
fails on 7.4.3.

* Add ParserOptions::clearStaticCache(), which resets static cached
  values back to their pre-hook values.
* Add accessors for these private static variables to protect against
  access before initialisation.
* Rename "inCache" to "cacheVaryingOptionsHash" for clarity.
* In ParserOptions::match(), filter static properties by detecting them
  with ReflectionProperty, instead of making a big list of them.
* Update ParserOptionsTest for the fact that stubthreshold is no longer
  a cache-varying option. This was hidden by
  ParserOptionsTest::clearCache() injecting the wrong array into
  ParserOptions.

Bug: T270228
Change-Id: I6b5ba022e1b889a10f9fbe93da63da9831504df8
2021-03-02 13:03:58 +11:00
Arlo Breault
c44a3958a3 Don't apply French spacing in raw text elements
This also means we don't need to take special care for French spacing in
attributes, since it's no longer applied there.

Adds a test that captures this change.

Note that the test "Nowiki and french spacing" wonders whether this
escaping should be applied to nowiki content.

Bug: T255007
Change-Id: Ic8965e81882d7cf024bdced437f684064a30ac86
2021-02-16 19:26:29 -05:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Umherirrender
a1de8b8700 Tests: Mark more more closures as static
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208

Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
2021-02-09 02:55:57 +00:00
Reedy
729f20afc8 Tests: Mark more closures as static
Bug: T274036
Change-Id: I911d3041cebe417d162934223b46ea295c6d20e3
2021-02-07 13:26:56 +01:00
Umherirrender
62002cdcf1 build: Update mediawiki/mediawiki-codesniffer to 35.0.0
Change-Id: Idb413be4b8cba8611afdc022af59810ce1a4531e
2021-01-31 13:34:38 +00:00
jenkins-bot
eb4b304ed8 Merge "Add missing @param and @return to documentation in tests" 2021-01-30 15:22:33 +00:00
Petr Pchelko
b964bd2e8d Make ParserOptions::isSafeToCache more robust
Bug: T273120
Change-Id: I516133d07d9ff4d7930de68b88ed4b40ca1c6150
2021-01-27 18:34:26 -06:00
jenkins-bot
d491f23b90 Merge "Respect used options for ParserOptions::isSafeToCache" 2021-01-25 19:13:53 +00:00
Umherirrender
7691dbeca9 Add missing @param and @return to documentation in tests
Change-Id: Ic663e81cca0bf007804a70772250914a85f1fef4
2021-01-22 19:57:25 +01:00
Petr Pchelko
816e02ae51 Convert RevisionRecord to Authority and PageIdentity
As we convert the RevisionRecord to using Authority,
we no longer need Title instances, so we can convert
that to PageIdentity.

Ideally, we'd part away from using Title at all, but:
1. For foreign wikis PageIdentity has stronger validation,
so calling PageIdentity getId() on Title will break things.
There's still a lot of code depending on lax Title guarantees,
so we keep it.
2. A lot of code still depends on Title, so we try to pass it
through even if we don't nesessarily need to, to save cost
on recreating it later on.

Bug: T271458
Depends-On: I287400b967b467ea18bebbb579e881a785a19158
Change-Id: I63d9807264d7e2295afef51fc9d982447f92fcbd
2021-01-21 13:37:01 -06:00
daniel
61a5531e4a Strict return types for PageIdentity
Bug: T208776
Change-Id: I7e71ffc209465b7cca385603d844cd40178485a9
2021-01-18 17:33:14 +01:00
jenkins-bot
7200e36c9f Merge "parser: convert Preprocessor to WANCache and inject dependencies" 2021-01-12 05:04:47 +00:00
Aaron Schulz
de6eeead21 parser: convert Preprocessor to WANCache and inject dependencies
Make the caching logic use getWithSetCallback() and simplify
the code given that there is only one Preprocessor subclass.
Also, keep the cached values JSON serializable but rely on
the serialization in BagOStuff instead for simplicity.

Add related class constants for injecting preprocessor flags.

Bug: T254608
Change-Id: I72f9f0c0bc352ed5120469090c71294ff0c24999
2021-01-11 20:21:10 -08:00
Petr Pchelko
46b66f093a Respect used options for ParserOptions::isSafeToCache
Bug: T269293
Change-Id: Ic3cf908265ad470815f0ac81442d33bde04a5665
2021-01-04 10:32:34 -06:00
DannyS712
334fb503d1 ParserPreloadTest: Remove unneeded unset calls
Change-Id: Ia21fac8232e79444ecfae47c1a53a81a08ef8395
2020-12-28 22:10:53 +00:00
DannyS712
6a93b0ca93 More misc test cleanup
* parent::setUp() should be first, and ::tearDown()
  should be last
* Move tests that directly extend PHPUnit\Framework\TestCase
  to /unit

Change-Id: I1172855c58f4f52a8f624e6d596ec43beb8c93ff
2020-12-24 00:52:06 +00:00
jenkins-bot
add80561d2 Merge "ParserOptions: Deprecate fallback to $wgUser" 2020-12-16 15:17:05 +00:00
Petr Pchelko
71bb51ed55 ParserCache: general code cleanup, abstracted expiration checks.
Change-Id: I7374f30d582064236b8f782e6a2528eb692e3010
2020-12-16 12:09:55 +00:00
DannyS712
25db391d53 ParserOptions: Deprecate fallback to $wgUser
Emit deprecation notices in the constructor
if falling back to $wgUser, and fix core
calls

Bug: T246861
Depends-On: I51117931d527a3bdda468b48de577a7faafbcd69
Change-Id: Ibd0a8ffd0494c17a378cc43e6b6164166130adf4
2020-12-16 09:03:29 +00:00
daniel
637f630fe9 Implement caching for old revision HTML endpoint
Bug: T269663
Change-Id: I2d17ec37d25f3a6e1c4836c05576bf0fabb7d429
2020-12-15 23:40:08 +01:00
daniel
00a3439dce Introduce RevisionOutputCache
Bug: T267981
Change-Id: Ib1dc641ed10d786918362b25bd655780d5844ba1
2020-12-14 16:50:28 +00:00
Petr Pchelko
66cc685b45 Make ParserCache use CachedBagOStuff
Bug: T269593
Change-Id: I21e6e39eccad22b781252b142c1e5b079c1ee0b4
2020-12-07 10:28:30 -06:00
Petr Pchelko
4417b13d58 Make ParserCache respect ParserOptions::isSafeToCache
Bug: T269154
Change-Id: I8e9ecd2787aa8d172e708ba64ea936e63fbc6b36
2020-12-02 14:02:36 -06:00
Máté Szabó
40d50dd2fc tests: Fix method signature mismatches
These cause fatals on PHP 8.

Bug: T248925
Change-Id: Ifc8536e99f4676180d47df483e55c89e9f834c32
2020-11-26 19:02:57 +01:00
Petr Pchelko
b956c77d27 Merge CacheTime and ParserOutput accessedOptions properties
Change-Id: I5785596d68e8923f8bcbd182ace0b1991bd75c9a
2020-11-19 10:12:39 -07:00
Petr Pchelko
dbdc2a3cd3 Introduce JsonCodec to help with serialization/deserialization
Change-Id: I5433090ae8e2b3f2a4590cc404baf838025546ce
2020-11-19 08:32:21 -07:00
Petr Pchelko
7c68ae9296 Safe ParserOutput extension data and JsonUnserializable helper.
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.

Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
2020-11-10 11:21:09 -07:00
jenkins-bot
646dd3d594 Merge "Introduce ParserOutputAccess" 2020-11-10 14:56:12 +00:00
daniel
67d0986211 Introduce ParserOutputAccess
Encapsulate logic for getting rendered page content, for any revision,
with caching and pooling hidden away.

Introducing such a service object will also give us a leverage point for
supporting output transformations. Output transformations are currently
implemented partially in ParserOutput, partially in Parser, and partially
duplicated in Parsoid.

Bug: T267234
Change-Id: I566d7a7936633823ba68b5aecbc8c2d88949b4f8
2020-11-10 15:12:12 +01:00
Petr Pchelko
017cfcf016 Forward-compat for merging CacheTime and ParserOutput mOptions
CacheTime::mUsedOptions and ParserOutput::mAccessedOptions
do exactly the same thing and has to be merged into a single property.
This patch adds forward-compatibility and needs to be deployed
at least one train before the patch which actually merges the properties.

Change-Id: Ic9d71a443994e2545ebf2a826b9155c82961cb88
2020-11-10 07:09:41 -07:00
daniel
cac89b547c ParserOutput: add support for binary properties in JSON.
This introduces a mechanism for encoding binary data in
strings set via setProperty(). This is needed to accommodate compressed
data as used by TemplateData, which uses gzip compression to make the
data fit into the page_props table.

Bug: T266200
Change-Id: I19fa0dea8c25d93fcdec9dc5ddd6f3c9c162b621
2020-11-04 18:52:09 +01:00
jenkins-bot
831fcaf0da Merge "Use ::class together with createMock in unit tests" 2020-10-30 17:56:24 +00:00
Petr Pchelko
2ab1aa3b0a Add some more tests for invalid JSON in ParserCache.
Change-Id: I4983592b9a964f4371ef42c824090468eb938862
2020-10-30 14:02:20 +00:00
Umherirrender
5d41326891 Use ::class together with createMock in unit tests
This makes it easier for IDEs to find usage
This works even for non-existing classes

Change-Id: I4a6389a9bc0b3c212633841d69bd4f48a7ed6f56
2020-10-30 14:45:37 +01:00