Commit graph

20 commits

Author SHA1 Message Date
C. Scott Ananian
46acc1a257 Use uniform representation for ParserOutput "index policy"
As part of the project of enforcing uniform semantics for
combining ParserOutput objects (T300979) use standard boolean flags
for the 'index' and 'noindex' index policy metadata.

The forward-compatibility "1.39_wmf.7-ParserCache-*" serialization
test cases have been renamed to "1.39-ParserCache-*" in this commit;
backward compatibility with the prior representation of index policy
will continue to be tested via the "1.38-ParserCache-*" cases.

Bug: T300979
Change-Id: I683e5ae054a0425b03c60a4af8c845b576414c1d
2022-03-31 09:51:21 -04:00
C. Scott Ananian
664012e23f Ensure forward compatibility with new ParserOutput index policy fields
Instead of ParserOutput::$mIndexPolicy, a future MW version
(I683e5ae054a0425b03c60a4af8c845b576414c1d) will use two boolean fields
ParserOutput::$mIndexSet and ::$mNoIndexSet.  For parser cache migration
purposes, ensure that core can deserialize the new version so that
rollback are safe.

Add serialization test cases with the new boolean fields as
"1.39_wmf.7-ParserOutput-*"; compatibility with the existing
"mIndexPolicy" serialization will continue to be tested with the
"1.38-ParserOutput-*" cases.

Change-Id: I5e4fc68cea18b31ecb028b3867537dcbd86b93cd
2022-03-31 09:51:21 -04:00
C. Scott Ananian
ea4bedee9d Remove misleading test in SerializationTestTrain::validateObjectEquality()
The initial object comparison using `==` compares all the property values
using `==` as well.  This causes (for example) the string "1" to compare
as equal to the number `1`, the empty string `""` to compare as equal
to the number `0` and other surprising things.

It appears that this comparison was done at an attempt at performance
optimization, but that has little place in a test suite.  Use the full
recursive strict comparison instead.

This requires updating one ParserCache test case which was apparently
created incorrectly (or not properly updated) and relied on the
previous behavior (numbers comparing as equal to strings) to pass.

Change-Id: Ife0e9ccc7be0f4933975bb326203693bc15a9658
2022-03-31 09:50:51 -04:00
jenkins-bot
99dee6855a Merge "Change return value of ParserOutput::getPageProperty() when property is missing" 2022-02-19 00:49:48 +00:00
C. Scott Ananian
c39ef6c6c9 Change return value of ParserOutput::getPageProperty() when property is missing
The old ParserOutput::getProperty() method returned `false` when a property
was missing.  This requires callers to use the `?:` syntax to supply default
values, which then causes any falsey value to be treated as missing.
So, for example, setting the defaultsort to '0' will cause the default
sort to be ignored.

Modern php convention is to use `null` for missing values, and the `??`
syntax is a better/more restrictive alternative to `?:`.

We renamed `ParserOutput::getProperty()` to `::getPageProperty()` in
1.38 (Ie963eea5aa0f0e984ced7c4dfa0fd65d57313cfa/T287216) but kept the
return value convention.  Before this actually makes it into a 1.38
release, take the opportunity to fix the return value for the new
`ParserOutput::getPageProperty()` method to return `null` when the
property is missing.

We need to do some temporary workarounds to the places we'd
already swapped over to use the new `::getPageProperty()` method
to allow them to handle either `false` or `null` as a return value;
we'll clean that up once this is merged.

Code search:
https://codesearch.wmcloud.org/deployed/?q=-%3EgetPageProperty%5C%28|T301915&i=nope&files=&excludeFiles=&repos=

Bug: T301915
Depends-On: I3f11ce604970e47b41fc1c123792df8c3045626f
Depends-On: Ie7533f49fe4cad01ebfda29760d23c61e9867b10
Depends-On: Ic5c09f5caa4c897bc553c614fbae9cee159566a2
Depends-On: I0278b2eafd90e77e4fee41c45a1165fb79ddf47e
Depends-On: I383abb6b7dc5e96c0061af13957609f6e31a1065
Depends-On: I79f9f4078e415284af29b15047bafd1c823d7f5b
Depends-On: I02276c48c49f5d2d241a69eb0a6cdf439b572d8b
Depends-On: I71628661b4539a4e35ae32846e719f92bcf782e0
Depends-On: I7e215cb43de0ce150a6bcc00f92481dcdcfed383
Change-Id: Iaa25c390118d2db2b6578cdd558f2defd5351d15
2022-02-18 21:15:58 +00:00
C. Scott Ananian
3c211fdb3c Update ParserCache serialization test cases to use valid category keys
Category keys are supposed to be non-null strings.

The test cases use bogus integer values, which causes issues when
refactoring more strictly enforces validity checks on category sort
key values.

Change-Id: If2937a694ba6bd4c522336f33aa58d40949b5a54
2022-02-17 12:12:53 -05:00
C. Scott Ananian
b46dfcc351 Update ParserCache serialization test cases to use a valid index policy
Valid values for ParserCache::$mIndexPolicy are '', 'noindex', and 'index'.

The test cases use the bogus value 'policy1', which causes issues when
refactoring more strictly enforces validity checks on index policy
values.

Change-Id: I2d00ff4e3ded043d18942c8482a39fac14ec60bc
2022-02-09 12:47:27 -05:00
Ppchelko
643fc535c3 Reapply "Move limit report rendering to ParserOutput"
This reverts commit 2bcb3fe567.

Reason for revert: this is a good change,
just needed more work to not break CI

Change-Id: I23768bee242e3cf81b1493a740cf070e7ad1e224
2021-11-09 11:08:08 -08:00
Petr Pchelko
a1aa3e0827 Hard-deprecate all public property access on CacheTime and ParserOutput.
- Added a test where ParserOutput objects with CacheTime
properties set are unserialized from previous versions.
- Generate new serialization tests for 1.38

Now all serialization in production is JSON, so changing
property visibility shouldn't affect ParserCache.

Bug: T263851
Depends-On: I283340ff559420ceee8f286ba3ef202c01206a23
Change-Id: I70d6feb1c995a0a0f763b21261141ae8ee6dc570
2021-10-13 13:27:16 -04:00
Petr Pchelko
2010a912c2 CacheTime: remove mVersion field
Has to be merged at least one train after the parent patch.

Bug: T269393
Change-Id: Iee98950ad0452a50088e755252da8468f0bec1ae
2021-01-04 10:29:52 -06:00
Daniel Kinzler
48172f794d Revert "Hard-deprecate all public property access on CacheTime and ParserOutput."
This reverts commit b1a30eb0c4.

Reason for revert: T269396

Bug: T269396
Change-Id: I374ca13ccc30418b8fe3bf98f5090f7643aac4d7
2020-12-04 11:47:55 +00:00
Petr Pchelko
b1a30eb0c4 Hard-deprecate all public property access on CacheTime and ParserOutput.
Bug: T263851
Change-Id: I3d3ff7b5b6899150df836e445b56896dfd5b887e
2020-11-19 10:12:41 -07:00
Petr Pchelko
b956c77d27 Merge CacheTime and ParserOutput accessedOptions properties
Change-Id: I5785596d68e8923f8bcbd182ace0b1991bd75c9a
2020-11-19 10:12:39 -07:00
Petr Pchelko
dbdc2a3cd3 Introduce JsonCodec to help with serialization/deserialization
Change-Id: I5433090ae8e2b3f2a4590cc404baf838025546ce
2020-11-19 08:32:21 -07:00
Petr Pchelko
7c68ae9296 Safe ParserOutput extension data and JsonUnserializable helper.
One major difference with what we've had before is that now we
actually write class names into the serialization - given that
this new mechanism is extencible, we can't establish any kind
of mapping of allowed classes. I do not think it's a problem
though.

Bug: T264394
Change-Id: Ia152f3b76b967aabde2d8a182e3aec7d3002e5ea
2020-11-10 11:21:09 -07:00
Petr Pchelko
017cfcf016 Forward-compat for merging CacheTime and ParserOutput mOptions
CacheTime::mUsedOptions and ParserOutput::mAccessedOptions
do exactly the same thing and has to be merged into a single property.
This patch adds forward-compatibility and needs to be deployed
at least one train before the patch which actually merges the properties.

Change-Id: Ic9d71a443994e2545ebf2a826b9155c82961cb88
2020-11-10 07:09:41 -07:00
daniel
cac89b547c ParserOutput: add support for binary properties in JSON.
This introduces a mechanism for encoding binary data in
strings set via setProperty(). This is needed to accommodate compressed
data as used by TemplateData, which uses gzip compression to make the
data fit into the page_props table.

Bug: T266200
Change-Id: I19fa0dea8c25d93fcdec9dc5ddd6f3c9c162b621
2020-11-04 18:52:09 +01:00
daniel
600f64029f Use JSON for parser cache
This adds JSON serialization and deserialization capabilities
to CacheTime and ParserOutput.

NOTE: JSON serialization is disabled for now. Merging this patch
should not change behavior in production.

Bug: T263579
Change-Id: I18187e8bce573d21f6f1bd29106e07c63a6d2f4d
2020-10-13 16:28:52 -07:00
Petr Pchelko
8a879605d9 Add deserialization acceptance tests for ParserOutput
Bug: T264397
Change-Id: I6476fd9b8eff0e1b61ce5f43280d1cd9b7aaa77c
2020-10-12 08:55:32 +00:00
daniel
6eea7d7ed5 Add test infra for ParserCache serialization/deserialization
Based on Daniel's work at Ia6e70179b7ee5ce4e93888585ccc30d92da165c3
however was changed enough to move into a separate changeset.

More acceptance tests and data will be added in a followup commit.

Bug: T264397
Change-Id: I135187e83cbfa02b97c5656f0752f8bf1ceb58d0
2020-10-09 08:14:57 -06:00