It's possible that this has some side effect on HipHop builds -- the require_once on UtfNormalDefines.php that had been moved from UtfNormalUtil.php to MediaWiki's Setup.php in r85944 had been previously damaged by r85327 sticking a MediaWiki-specific MWInit class static method call into it, making it break on all the generator & test & benchmark code which does not depend on MediaWiki.
With the require_once restored everything seems to work, though it's possible that it ends up trying to include twice on HipHop -- have not tested.
fed an invalid sequence in UtfNormal::cleanUp
normalizer_normalize seems to return false if fed an invalid unicode sequence (Which is quite different
from what our built in normalization functions do). So use quickIsNFC if it returns false.
(Noticed when investigating bug 28541).
* Work around HipHop issue 314 (volatile broken) and issue 308 (no compilation detection) by adding some large and ugly compilation detection code to WebStart.php and doMaintenance.php.
* Provide an MW_COMPILED constant which can be used to detect compiled mode throughout the codebase.
* Introduced wfIsHipHop(), which detects either compiled or interpreted mode. Used this to work around unusual eval() return value in eval.php.
* Work around lack of ini_get() in Maintenance.php, by duplicating wfIsHipHop().
* In Maintenance::shouldExecute(), accept "include" as an inclusion function name, since all kinds of inclusion give this string in HipHop.
* Introduced new class MWInit, which provides some static functions in the pre-autoloader environment.
* Introduced MWInit::compiledPath(), which provides a relative path for invoking a compiled file, and MWInit::interpretedPath(), which provides an absolute path for interpreting a PHP file. Used these new functions in the appropriate places.
* When we are running compiled code, don't include files which would generate duplicate class, function or constant definitions. Documented the new requirements on the contents of Defines.php and UtfNormalDefines.php.
* In HipHop compiled mode, it's not possible to have executable code in the same file as a class definition.
** Moved MimeMagic initialisation to the constructor.
** Moved Namespace.php global variable initialisation to Setup.php.
** Moved MemcachedSessions.php initialisation to the caller in GlobalFunctions.php.
** Moved Sanitizer.php constants and global variables to static class members. Introduced an accessor function for the attribs regex, as a new place to put code formerly at file level.
** Moved Language.php initialisation of $wgLanguageNames to Language::getLanguageNames(). Removed the global variable, marked "private" since forever.
* In two places: don't use error_log() with type=3 to append to a file, HipHop doesn't support it. Use file_put_contents() with FILE_APPEND instead.
* Work around the terrible breakage of class_exists() by using MWInit::classExists() instead in various places. In WebInstaller::getPageByName(), the class_exists() was marked with a fixme comment already, so I replaced it with an autoloader solution.
This pulls the source texts used for the UtfNormalBench.php benchmarks, repeats them into much larger strings, and attempts to reproduce an out-of-memory error in the middle of UtfNormal::cleanUp():
$ php
Testing testdata/washington.txt (English text)...
quickIsNFCVerify 1078.3ms 14,969,652 bytes/s (changed)
cleanUp 992.1ms 16,270,594 bytes/s (no change)
Testing testdata/berlin.txt (German text)...
PHP Fatal error: Allowed memory size of 136314880 bytes exhausted (tried to allocate 71 bytes) in /var/www/trunk/includes/normal/UtfNormal.php on line 285
Exact failure point may require adjustment depending on platform, etc.
* This involved updating of autogenerated Unicode normalisation tables, which are compliant with Unicode 5.2.0 now. All normalization tests pass except for RandomTest.php which have bitrotten to its death from PHP fatals.
* Added a line about r59036 to release notes
Doxygen documentation update:
* Changed alls @addtogroup to @ingroup. @addtogroup adds the comment to the group description, but doesn't add the file, class, function, ... to the group like @ingroup does. See for example http://svn.wikimedia.org/doc/group__SpecialPage.html where it's impossible to see related files, classes, ... that should belong to that group.
* Added @file to file description, it seems that it should be explicitely decalred for file descriptions, otherwise doxygen will think that the comment document the first class, variabled, function, ... that is in that file.
* Removed some empty comments
* Removed some ?>
Added following groups:
* ExternalStorage
* JobQueue
* MaintenanceLanguage
One more thing: there are still a lot of warnings when generating the doc.
Note that case mappings will only be used if mbstring extension is not present.
Normalization data files updated to Unicode 5.1.0; passes the automated tests.
Seem to have long since lost the script I originally used to generate the Utf8Case.php mapping file, which appears not to have been updated since 2002 or so. :)
Made a new one and moved it into the UtfNormal sub-library.
Note a couple limitations:
* Case mapping (still) uses only the 1:1 simple mappings. Any full or locale-specific mappings are ignored.
* These case mappings are not used anyway when the PHP mbstring extension is available; mbstring's case conversion functions are used instead, with whatever version of Unicode support and whatever complex mapping support they may or may not have.
* The generated Utf8Case.php file is not used directly -- you must also regenerate the serialized version in the 'serialized' directory after updating it to a new Unicode version.
Moved them out to a separate file in the library which can be cleanly included from both places for transparent happiness.
A fresh rebuild & test of UtfNormal data via 'make test' now works fine.
* Added support for configuration of an arbitrary number of commons-style file repositories.
* Split Image.php into filerepo/File.php and filerepo/LocalFile.php
* Renamed Image::getImagePath() to File::getPath()
* Added initial support for timestamp-based file fetching (OldLocalFile), to be expanded upon by aaron.
* Changed the interface for Image/File object creation: use wfFindFile() or wfLocalFile() depending on semantics
* ImageGallery::add() now accepts a title object as the first parameter
* Moved file handling operations on upload from SpecialUpload to File
* Removed path-related functions from ImageFunctions.php. Removed static path accessors from File.
* Added a Content-Disposition header to thumb.php output
* Improved thumb.php error handling
* Updated the unit test suite to kind of partially work with modern computers. RunTests.php doesn't work just yet. Fixed an actual regression that the test suite detected -- moved some defines to Defines.php where they will be loaded consistently.
* Seems like an opportune time to introduce "@addtogroup Media" documentation tags.
* Merge "@addtogroup Metadata" (used by Exif.php) into "@addtogroup Media".
* Few more moving comment blocks to above classes.
