Add FLD_VALUE_VERSION key to the value wrapper array to hold the version
number used in getWithSetCallback(). Remove the VFLD_* wrapper array from
FLD_VALUE for versioned values.
Keys stored with the old VFLD_VERSION and VFLD_DATA fields will be seen as
having the wrong version. The previous WAN cache code will see the new keys
that use FLD_VALUE_VERSION as having the wrong version too. In either case,
the usual variant key logic applies, so there should not be any issues.
This means that moving from a non-versioned to a versioned cache key is no
longer a breaking change when, for the same key, some code passes a version
number to getWithSetCallback() while other code does not.
Also:
* Make "pcTTL" respect the version number for sanity
* Make sure set() respects TTL_UNCACHEABLE for completeness
* Track slow regeneration callback runtime in FLD_GENERATION_TIME
* Remove is_callable() check overhead and rely on PHP Error instances
* Refactor unwrap() to return a more immediately useful value
* Simplify getNonProcessCachedKeys() signature by using $opts
* Split out PURGE_* constants for purge entries since those keys are
never stored in any serialize value but are only in PHP arrays
* Rename doGetWithSetCallback() to be more succinct
* Rename and reorganize some variables for clarity
Change-Id: I4060b19583cdfd9fa36c91d7014441eeef4b3609
In SqlBagOStuff:
* Add modifyMulti() helper method to reduce code duplication
* Improve atomicity of add(), cas(), and changeTTL() queries
* Avoid integer serialization and improve atomicity of incr()
* Optimize new BagOStuff::changeTTLMulti() method
In BagOStuff:
* Add changeTTLMulti() method for subclasses to optimize
* Make set() ignore WRITE_ALLOW_SEGMENTS for integers so incr() works
* Strip WRITE_ALLOW_SEGMENTS flag from the setMulti() call in set() to
avoid triggering bogus sanity check exceptions
* Fix BagOStuffTest::testSetSegmentable failures via the above changes
* Enforce WRITE_ALLOW_SEGMENTS sanity check in setMulti() for all the
subclasses by using a final wrapper method
* Add WRITE_ALLOW_SEGMENTS sanity check to deleteMulti()
Bug: T113916
Change-Id: I25d1790fa9b0d1837643efccfa94a12043cfbf42
Mediawiki does not recognizes the mime type of an OpenOffice / Libre
Office Database Frontend Document and thus uses application/zip.
Our MimeAnalyzer::detectZipType() looks at the Zip files to detect known
signatures, however it is based on a reference from 2005 which did not
have the Database type yet.
OASIS Open Document Format v1.2 specify non normative MIME types, the
reference supposedly being the ones registered with the IANA, but we
have some dispredancy: ODF recommends 'base' while the IANA got
'database' registered: https://www.iana.org/assignments/media-types/
I supposed the format got approved as is, the Database mime type being
'recommended' probably because the IANA registration was pending.
Add the 'base' type which is being used by OpenOffice 3.x and later as
well as at least Libre Office 5.x.
Document my findings in code comments.
Add a test.
Bug: T35515
Change-Id: If0210a87067358612ecb8b3edd001fb05d01653d
MSCompoundFileReader::readFile uses iconv to convert information
given in UTF-16 character set with little-endian byte order to
the UTF-8 character set. The input string has no BOM and the byte order
is not explicitly given, causing iconv to try to guess the byte order
based on the host operating system. This causes the method to return
different results for the same file in different environments.
This patch explicitly provides the byte order for the input to be
converted (UTF-16LE) to ensure portability and predictability.
As part of this, move MSCompoundFileReaderTest into the unit test tree.
Bug: T225019
Change-Id: I62154897d303b28c288c3a4f2f5456bedcc81852
Make flushSnapshot() logging more detailed and check for explicit transaction
rounds. Also removing periods and trailing newlines from log/exception messages.
Change-Id: I0f6520f563680ab3a65b6338ced59ba25a2ec7b5
This changeset resumes work on T89432 and related tickets
by porting an initial set of tests to the new unit test suite
separated out in I69b92db3e70093570e05cc0a64c7780a278b321a.
The tests were only ported if they worked immediately without
requiring any changes other than changing the test case class
to MediaWikiUnitTestCase and moving the test to the new suite.
If a test failed for any reason (even trivial misconfiguration),
it was NOT ported.
With this change, the unit tests suite now consits of a total
of 455 tests. As before, you can run these tests via the following
command:
$ composer phpunit:unit
Bug: T84948
Bug: T89432
Bug: T87781
Change-Id: Ibb8175981092d7f41864e641cc3c118af70a5c76
The callback will be called immediately when the section is cancelled,
whether that occurs directly, or via cancelling of an outer section, or
via rollback of the entire transaction.
Change-Id: Id05296948b52b95544547bd38c4387496b6c83b9
With the introduction of a REST API into MediaWiki core, we're going to
want to share parameter validation logic rather than having similar code
in both the Action API and the REST API. This abstracts out parameter
validation logic as a library.
There will be at least two follow-up patches:
* One to add calls in the REST API, plus the interface for the REST API
to do body validation. Should be reasonably straightforward.
* One to adjust the Action API to use this. That'll be much less
straightforward, as the Action API needs some MediaWiki-specific types
(which the REST API might use too in the future) and needs to override
the defaults on some of the library's checks (to maintain back-compat).
Bug: T142080
Bug: T223239
Change-Id: I5c0cc3a8d686ace97596df5832c450a6a50f902c
Depends-On: Iea05dc439688871c574c639e617765ae88a75ff7
This is needed in order for Phan not to consider calls to
IDatabase::buildLike as invalid. Interestingly, it does not
consider calls to Database::buildLike invalid.
Bug: T191668
Change-Id: I0e027f5ec66d20b1d11e3441086001f6a751e1f5
This changeset implements T89432 and related tickets and is based on exploration
done at the Prague Hackathon. The goal is to identify tests in MediaWiki core
that can be run without having to install & configure MediaWiki and its dependencies,
and provide a way to execute these tests via the standard phpunit entry point,
allowing for faster development and integration with existing tooling like IDEs.
The initial set of tests that met these criteria were identified using the work Amir did in
I88822667693d9e00ac3d4639c87bc24e5083e5e8. These tests were then moved into a new subdirectory
under phpunit/ and organized into a separate test suite. The environment for this suite
is set up via a PHPUnit bootstrap file without a custom entry point.
You can execute these tests by running:
$ vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml
Bug: T89432
Bug: T87781
Bug: T84948
Change-Id: Iad01033a0548afd4d2a6f2c1ef6fcc9debf72c0d
Use it for ApiStashEdit so that large PaserOutput can be stored.
Add flag to allow for value segmentation on set() in BagOStuff.
Also add a flag for immediate deletion of segments on delete().
BagOStuff now has base serialize()/unserialize() methods.
Bug: T204742
Change-Id: I0667a02612526d8ddfd91d5de48b6faa78bd1ab5
This shares reconnection and retry logic but lacks some of the
restrictions applied to queries that go through the public query()
interface.
Use this in a few places such as doSelectDomain() for mysql/mssql.
Bug: T212284
Change-Id: Ie7341a0e6c4149fc375cc357877486efe9e56eb9
Follow up to 8107fddd8f
Also improve debug logging and add some unit tests
Bug: T223310
Bug: T223978
Change-Id: I35484385e4da2912bc10f5e8d2fb07cb1097347e
This also moves several tests out of HttpTest, so they are not lost when
the deprecated Http class is removed.
Bug: T222935
Change-Id: Ib4aac96ba1a978e8b6738ed446663afc3cc8c792
This reduces confidence in the test. There is no guruantee that
it won't return the same value twice during the duration of a full
PHPUnit run of all test suites, whether twice in a row or 20 minutes
apart.
For a test that needs a string of any kind, use an explicit, consinstent
and cheap literal value.
For a test that specifically needs some kind of uniqueness compared to
something else within the same test case, do so explicitly.
Tests that require something globally unique (for some undefined/vague
definition of "global") were not found, and should not exist anyway.
Also, in libs/objectcache tests, fix order of parameters in some
assertions (expected first, then actual), and use assertFalse/assertSame
instead of assertEqual for cases where false is expected to remove
tolerance of other loosely equal values.
Change-Id: Ifc60e88178da471330b94bfbf12e2731d2efc77d
HHVM does not support variadic arguments with type hints. This is
mostly not a big problem, because we can just drop the type hint, but
for some reason PHPUnit adds a type hint of "array" when it creates
mocks, so a class with a variadic method can't be mocked (at least in
some cases). As such, I left alone all the classes that seem like
someone might like to mock them, like Title and User. If anyone wants
to mock them in the future, they'll have to switch back to
func_get_args(). Some of the changes are definitely safe, like
functions and test classes.
In most cases, func_get_args() (and/or func_get_arg(), func_num_args() )
were only present because the code was written before we required PHP
5.6, and writing them as variadic functions is strictly superior. In
some cases I left them alone, aside from HHVM compatibility:
* Forwarding all arguments to another function. It's useful to keep
func_get_args() here where we want to keep the list of expected
arguments and their meanings in the function signature line for
documentation purposes, but don't want to copy-paste a long line of
argument names.
* Handling deprecated calling conventions.
* One or two miscellaneous cases where we're basically using the
arguments individually but want to use them as an array as well for
some reason.
Change-Id: I066ec95a7beb7c0665146195a08e7cce1222c788
Even if there is only one database, if a DB_REPLICA connection
reference is obtained and an attempt is made to write it then
it will still fail. This can make it easier to spot incorrect
database usage even in a simple vanilla developer environment.
Note that methods like ILoadBalancer::getConnectionRef() can
be used for local connections as well as foreign ones.
Change-Id: I5523daad1bdd64624d3e0dd41bfd8d078b1078b0
This method easily fails due to use of the same resource in each
thread after calling pcntl_fork(). This can cause thread-safety
problems like mysqli errors, which in turn make tests fail.
Change-Id: I4ff3657940c1d1cdad58bafb7449bdd0182cbe56
Also, reorder some method definitions to match the base class.
This makes it easier to compare the two. In addition, refactor
the doWrite() method to make the calls clearer.
Change-Id: I795b395b9c54645f62962461a28067ae38b291ed
Make IDatabase::lastDoneWrites() reflect creation and changes to
the cloned temporary unit test tables but not other temporary tables.
This effects the LB method hasOrMadeRecentMasterChanges(). Other tables
are assumpted to really just be there for temporary calculations rather
acting as test-only ephemeral versions of permanent tables. Treating
writes to the "fake permanent" temp tables more like real permanent
tables means that the tests better align with production.
At the moment, temporary tables still have to use DB_MASTER, given
the assertIsWritableMaster() check in query(). This restriction
can be lifted at some point, when RDBMs compatibility is robust.
Bug: T218388
Change-Id: I4c0d629da254ac2aaf31aae35bd2efc7bc064ac6
Also:
* Fixed LoadBalancer::getAnyOpenConnection for both
DB_MASTER and DB_REPLICA, which are not real indexes.
* Lock down DBConnRef::close since it can only cause trouble.
* Relax DBConnRef restrictions on tablePrefix()/dbSchema()
for the harmless "getter" mode case.
* Remove redundant DatabasePostgres::getServer definition.
Change-Id: Ia855d901cc3c28147e52284fdabb1645805d4466
Also:
* Make the BagOStuff tests actually pass for memcached and sql
* Add more unit tests for BagOStuff
* Make SqlBagOStuff::add() more atomic
Change-Id: Ic1eec0990a66b595b57c646498c3bd229442230c
