Commit graph

481 commits

Author SHA1 Message Date
Arlo Breault
c44a3958a3 Don't apply French spacing in raw text elements
This also means we don't need to take special care for French spacing in
attributes, since it's no longer applied there.

Adds a test that captures this change.

Note that the test "Nowiki and french spacing" wonders whether this
escaping should be applied to nowiki content.

Bug: T255007
Change-Id: Ic8965e81882d7cf024bdced437f684064a30ac86
2021-02-16 19:26:29 -05:00
Umherirrender
8de3b7d324 Use static closures where safe to use
This is micro-optimization of closure code to avoid binding the closure
to $this where it is not needed.

Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I0ffc6200f6c6693d78a3151cb8cea7dce7c21653
2021-02-11 00:13:52 +00:00
Umherirrender
a1de8b8700 Tests: Mark more more closures as static
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208

Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
2021-02-09 02:55:57 +00:00
Reedy
729f20afc8 Tests: Mark more closures as static
Bug: T274036
Change-Id: I911d3041cebe417d162934223b46ea295c6d20e3
2021-02-07 13:26:56 +01:00
Umherirrender
62002cdcf1 build: Update mediawiki/mediawiki-codesniffer to 35.0.0
Change-Id: Idb413be4b8cba8611afdc022af59810ce1a4531e
2021-01-31 13:34:38 +00:00
jenkins-bot
eb4b304ed8 Merge "Add missing @param and @return to documentation in tests" 2021-01-30 15:22:33 +00:00
Petr Pchelko
b964bd2e8d Make ParserOptions::isSafeToCache more robust
Bug: T273120
Change-Id: I516133d07d9ff4d7930de68b88ed4b40ca1c6150
2021-01-27 18:34:26 -06:00
jenkins-bot
d491f23b90 Merge "Respect used options for ParserOptions::isSafeToCache" 2021-01-25 19:13:53 +00:00
Umherirrender
7691dbeca9 Add missing @param and @return to documentation in tests
Change-Id: Ic663e81cca0bf007804a70772250914a85f1fef4
2021-01-22 19:57:25 +01:00
Petr Pchelko
816e02ae51 Convert RevisionRecord to Authority and PageIdentity
As we convert the RevisionRecord to using Authority,
we no longer need Title instances, so we can convert
that to PageIdentity.

Ideally, we'd part away from using Title at all, but:
1. For foreign wikis PageIdentity has stronger validation,
so calling PageIdentity getId() on Title will break things.
There's still a lot of code depending on lax Title guarantees,
so we keep it.
2. A lot of code still depends on Title, so we try to pass it
through even if we don't nesessarily need to, to save cost
on recreating it later on.

Bug: T271458
Depends-On: I287400b967b467ea18bebbb579e881a785a19158
Change-Id: I63d9807264d7e2295afef51fc9d982447f92fcbd
2021-01-21 13:37:01 -06:00
daniel
61a5531e4a Strict return types for PageIdentity
Bug: T208776
Change-Id: I7e71ffc209465b7cca385603d844cd40178485a9
2021-01-18 17:33:14 +01:00
jenkins-bot
7200e36c9f Merge "parser: convert Preprocessor to WANCache and inject dependencies" 2021-01-12 05:04:47 +00:00
Aaron Schulz
de6eeead21 parser: convert Preprocessor to WANCache and inject dependencies
Make the caching logic use getWithSetCallback() and simplify
the code given that there is only one Preprocessor subclass.
Also, keep the cached values JSON serializable but rely on
the serialization in BagOStuff instead for simplicity.

Add related class constants for injecting preprocessor flags.

Bug: T254608
Change-Id: I72f9f0c0bc352ed5120469090c71294ff0c24999
2021-01-11 20:21:10 -08:00
Petr Pchelko
46b66f093a Respect used options for ParserOptions::isSafeToCache
Bug: T269293
Change-Id: Ic3cf908265ad470815f0ac81442d33bde04a5665
2021-01-04 10:32:34 -06:00
DannyS712
334fb503d1 ParserPreloadTest: Remove unneeded unset calls
Change-Id: Ia21fac8232e79444ecfae47c1a53a81a08ef8395
2020-12-28 22:10:53 +00:00
DannyS712
6a93b0ca93 More misc test cleanup
* parent::setUp() should be first, and ::tearDown()
  should be last
* Move tests that directly extend PHPUnit\Framework\TestCase
  to /unit

Change-Id: I1172855c58f4f52a8f624e6d596ec43beb8c93ff
2020-12-24 00:52:06 +00:00
jenkins-bot
add80561d2 Merge "ParserOptions: Deprecate fallback to $wgUser" 2020-12-16 15:17:05 +00:00
Petr Pchelko
71bb51ed55 ParserCache: general code cleanup, abstracted expiration checks.
Change-Id: I7374f30d582064236b8f782e6a2528eb692e3010
2020-12-16 12:09:55 +00:00
DannyS712
25db391d53 ParserOptions: Deprecate fallback to $wgUser
Emit deprecation notices in the constructor
if falling back to $wgUser, and fix core
calls

Bug: T246861
Depends-On: I51117931d527a3bdda468b48de577a7faafbcd69
Change-Id: Ibd0a8ffd0494c17a378cc43e6b6164166130adf4
2020-12-16 09:03:29 +00:00
daniel
637f630fe9 Implement caching for old revision HTML endpoint
Bug: T269663
Change-Id: I2d17ec37d25f3a6e1c4836c05576bf0fabb7d429
2020-12-15 23:40:08 +01:00
daniel
00a3439dce Introduce RevisionOutputCache
Bug: T267981
Change-Id: Ib1dc641ed10d786918362b25bd655780d5844ba1
2020-12-14 16:50:28 +00:00
Petr Pchelko
66cc685b45 Make ParserCache use CachedBagOStuff
Bug: T269593
Change-Id: I21e6e39eccad22b781252b142c1e5b079c1ee0b4
2020-12-07 10:28:30 -06:00
Petr Pchelko
4417b13d58 Make ParserCache respect ParserOptions::isSafeToCache
Bug: T269154
Change-Id: I8e9ecd2787aa8d172e708ba64ea936e63fbc6b36
2020-12-02 14:02:36 -06:00
Máté Szabó
40d50dd2fc tests: Fix method signature mismatches
These cause fatals on PHP 8.

Bug: T248925
Change-Id: Ifc8536e99f4676180d47df483e55c89e9f834c32
2020-11-26 19:02:57 +01:00
Petr Pchelko
b956c77d27 Merge CacheTime and ParserOutput accessedOptions properties
Change-Id: I5785596d68e8923f8bcbd182ace0b1991bd75c9a
2020-11-19 10:12:39 -07:00
Petr Pchelko
dbdc2a3cd3 Introduce JsonCodec to help with serialization/deserialization
Change-Id: I5433090ae8e2b3f2a4590cc404baf838025546ce
2020-11-19 08:32:21 -07:00
Petr Pchelko
7c68ae9296 Safe ParserOutput extension data and JsonUnserializable helper.
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.

Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
2020-11-10 11:21:09 -07:00
jenkins-bot
646dd3d594 Merge "Introduce ParserOutputAccess" 2020-11-10 14:56:12 +00:00
daniel
67d0986211 Introduce ParserOutputAccess
Encapsulate logic for getting rendered page content, for any revision,
with caching and pooling hidden away.

Introducing such a service object will also give us a leverage point for
supporting output transformations. Output transformations are currently
implemented partially in ParserOutput, partially in Parser, and partially
duplicated in Parsoid.

Bug: T267234
Change-Id: I566d7a7936633823ba68b5aecbc8c2d88949b4f8
2020-11-10 15:12:12 +01:00
Petr Pchelko
017cfcf016 Forward-compat for merging CacheTime and ParserOutput mOptions
CacheTime::mUsedOptions and ParserOutput::mAccessedOptions
do exactly the same thing and has to be merged into a single property.
This patch adds forward-compatibility and needs to be deployed
at least one train before the patch which actually merges the properties.

Change-Id: Ic9d71a443994e2545ebf2a826b9155c82961cb88
2020-11-10 07:09:41 -07:00
daniel
cac89b547c ParserOutput: add support for binary properties in JSON.
This introduces a mechanism for encoding binary data in
strings set via setProperty(). This is needed to accommodate compressed
data as used by TemplateData, which uses gzip compression to make the
data fit into the page_props table.

Bug: T266200
Change-Id: I19fa0dea8c25d93fcdec9dc5ddd6f3c9c162b621
2020-11-04 18:52:09 +01:00
jenkins-bot
831fcaf0da Merge "Use ::class together with createMock in unit tests" 2020-10-30 17:56:24 +00:00
Petr Pchelko
2ab1aa3b0a Add some more tests for invalid JSON in ParserCache.
Change-Id: I4983592b9a964f4371ef42c824090468eb938862
2020-10-30 14:02:20 +00:00
Umherirrender
5d41326891 Use ::class together with createMock in unit tests
This makes it easier for IDEs to find usage
This works even for non-existing classes

Change-Id: I4a6389a9bc0b3c212633841d69bd4f48a7ed6f56
2020-10-30 14:45:37 +01:00
Thiemo Kreuz
1fc8d79ac6 Remove documentation that literally repeats the code
For example, documenting the method getUser() with "get the User
object" does not add any information that's not already there.
But I have to read the text first to understand that it doesn't
document anything that's not already obvious from the code.

Some of this is from a time when we had a PHPCS sniff that was
complaining when a line like `@param User $user` doesn't end
with some descriptive text. Some users started adding text like
`@param User $user The User` back then. Let's please remove
this.

Change-Id: I0ea8d051bc732466c73940de9259f87ffb86ce7a
2020-10-27 19:20:26 +00:00
Petr Pchelko
8cc6b7f99a ParserCache JSON - do not \u encode unicode and special characters.
Without passing ALL_OK constant, json-encoding will \u-escape
all the unicode, which will blow the size of serialized data,
especially on Russian wiki out of proportion.

Bug: T263579
Change-Id: Ifaaf1cdfaeeb17c3a99ed742b64ae5cc3157500c
2020-10-22 18:26:59 -07:00
DannyS712
e2731a76ad Normalize error messages for non-serializable properties
Change-Id: If599082bd4acdc9df5b32aaabf2ba8d24e830914
2020-10-21 22:49:57 +00:00
Petr Pchelko
09c14b9dd0 Move serializability validation from ParserOutput to ParserCache
Bug: T263579
Change-Id: Iac2dbc817c2e7af4a6d112f01bd380a04354db22
2020-10-15 13:15:30 -07:00
daniel
600f64029f Use JSON for parser cache
This adds JSON serialization and deserialization capabilities
to CacheTime and ParserOutput.

NOTE: JSON serialization is disabled for now. Merging this patch
should not change behavior in production.

Bug: T263579
Change-Id: I18187e8bce573d21f6f1bd29106e07c63a6d2f4d
2020-10-13 16:28:52 -07:00
Petr Pchelko
bb39896603 Hard-deprecate ParserCache::getKey.
Bug: T263689
Depends-On: I20b5a3eece79afaac6a4fef733d7a60ea23c6ffe
Depends-On: I3ed1188e267f4eaab0ae46f2bc6f9a379dea58ce
Change-Id: I30d05ee5b217fce0521d14867309979e76f34760
2020-10-13 08:31:23 -07:00
Petr Pchelko
13574e8404 Deprecate ParserCache::getKey and replace it with getMetadata
Bug: T263689
Change-Id: I4a71e5a7eb1c25cd53b857c115883cd00160736b
2020-10-13 08:31:22 -07:00
Petr Pchelko
8a879605d9 Add deserialization acceptance tests for ParserOutput
Bug: T264397
Change-Id: I6476fd9b8eff0e1b61ce5f43280d1cd9b7aaa77c
2020-10-12 08:55:32 +00:00
daniel
6eea7d7ed5 Add test infra for ParserCache serialization/deserialization
Based on Daniel's work at Ia6e70179b7ee5ce4e93888585ccc30d92da165c3
however was changed enough to move into a separate changeset.

More acceptance tests and data will be added in a followup commit.

Bug: T264397
Change-Id: I135187e83cbfa02b97c5656f0752f8bf1ceb58d0
2020-10-09 08:14:57 -06:00
daniel
ff07253be5 ParserCache: be resilient to string values
This makes the parser cache resilient to encountering string values
where it is currently expecting to get a ParserOutput objerct from the
underlying cache.

This provides forward compatibility with a switch to JSON based caching:
If we have to switch back after writing JSON to the cache for a while,
ParserCache would simply ignore the respective entries, rather than
causing fatal errors.

Bug: T263579
Change-Id: Iaed582097ab2d05edb4b99a738ac39c530fd63c1
2020-10-01 14:53:00 -06:00
Petr Pchelko
e7ff3cbb6b Cover ParserCache with integration tests
Bug: T250500
Change-Id: I8c45e7c6706b532f1569d06330cc45e841f208b7
2020-10-01 13:56:22 -06:00
Ppchelko
3254e41a4c Revert "Revert "Revert "Hard deprecate all public properties in CacheTime and ParserOutput"""
This reverts commit deacee9088.

Bug: T264257
Change-Id: Ie68d8081a42e7d8103e287b6d6857a30dc522f75
2020-10-01 12:03:41 -06:00
jenkins-bot
da3270feac Merge "Hard-deprecate passing -1 to CacheTime::setCacheTime" 2020-09-28 18:25:11 +00:00
jenkins-bot
b43b4c728f Merge "Revert "Revert "Hard deprecate all public properties in CacheTime and ParserOutput""" 2020-09-24 16:26:17 +00:00
Ppchelko
deacee9088 Revert "Revert "Hard deprecate all public properties in CacheTime and ParserOutput""
This reverts commit a4dc6d82af.

I've reverted the merged patch since I didn't do enough testing
on serialized/reserialized ParserOutput and CacheTime. Now I'm
confident serialization/deserialization works.

Changes since original reverted version:
 - Use __get/__set instead of DeprecationHelper in order to
   avoid $deprecateProperties array to be serialized.
 - Add test for old format serialization new format deserialization.

Change-Id: Ic911c2724ad709931d3316e609781fb89b5b7b28
2020-09-24 07:55:18 -07:00
Petr Pchelko
6a5eb040c7 Hard-deprecate passing -1 to CacheTime::setCacheTime
I've been trying to find how -1 can actually get to
the cacheTime and could find any usages in core or
extensions. The proper way of making ParserOutput
uncacheable is to use updateExpiry( 0 ), that is
documented and has some warning about performance costs.
Setting the cacheTime to -1 seems to be an undocumented
feature (it is mentioned in comments as undocumented)
that doesn't seem to be used anywhere anymore.

I intend to remove it entirely after the deprecation is
deployed and we so not see any warnings for a few weeks.

Change-Id: Ie90b7e4a21faae726940fa9082f2e6a6ea8df613
2020-09-23 19:48:51 -07:00