Replace the remaining instances of Title with TitleValue, and use
services from MediaWikiServices instead of calling deprecated singleton
functions.
Change-Id: I5d13939a76380fff6b787cea8d4a5f90c1a31a5d
This fixes three issues with the installer:
1) Make sure LocalizationCache can find the installer's i18n files.
2) Make sure we don't try to use an SqlBagOStuff for caching before we have
a functioning database.
3) Don't try to output HTML when redirecting (this is unrelated to
MediaWikiServices, but came up during testing)
Bug: T135169
Change-Id: I7caa932024cd771d6fa226a3ac6001c3148ecc9c
LinkCache::singleton() is now deprecated, and the destroySingleton() and
setSingleton() methods were removed. They were not used in extensions,
and the usage in core was updated to use MediaWikiServices.
Change-Id: I08bb4f7913b03f71331ff683d0197c948aad6790
* Timeouts happen when memcached is cleared or on DC-switchover likewise
* Disable the sidebar cache when the MessageCache is disabled
* Also lower any output expiry for CDN and set a custom header
when the MessageCache is disabled
* Log when this happens to a dedicated "MessageCacheError" channel
Bug: T133069
Bug: T7092
Change-Id: I1b80e250532033c52680246279e1e8f56350ae83
This also corrects a bug in MessageCache::parse() where it takes a
language code string but the Parser expects a Language object in
ParserOptions.
Bug: T128809
Change-Id: I3508c9e515e505890b3c62bc4a0982aa2a313782
Can cause fatal if your database has invalid titles
stored in it (Title::makeTitleSafe is fed to $lb->addObj()
without checking that the result is non-null)
This reverts commit 0e5a1f8879.
Change-Id: I4174b2fdcc3d8e24ecf0bcc39cd0f0f2e8bfd6b2
If the Title object isn't the title of the current viewed WikiPage, the page_lang
field of the database isn't requested. This results in the problem, that
Title::getPageLanguage() always returns the default content language, even if
the page language is different (changed with Special:PageLanguage, if
wgPageLanguageUseDB is true). That is problematic for the Translate extension,
which relies on the correct page language.
This change makes sure, that getPageLanguage() always return the correct page
language. If the page language isn't loaded already, Title::getPageLanguage()
now does a database lookup (if $wgPageLanguageUseDB is true) to get the correct
page language. It will use LinkCache for the page_lang field.
Bug: T121666
Change-Id: I0ae5ea39f7a124ed427ca5dfb26c1a116b27a94e
The plan here is to take it out of 1.27.0-wmf.12 and put it back in
1.27.0-wmf.13.
Since BotPasswords depends on SessionManager, that's getting temporarily
removed too.
This reverts the following commits:
* 6acd424e0d SessionManager: Notify AuthPlugin before calling hooks
* 4d1ad32d8a Close a loophole in CookieSessionProvider
* fcdd643a46 SessionManager: Don't save non-persisted sessions to backend storage
* 058aec4c76 MessageCache: Don't get a ParserOptions for $wgUser before the end of Setup.php
* b5c0c03bb7 SessionManager: Save user name to metadata even if the user doesn't exist locally
* 13f2f09a19 SECURITY: Fix User::setToken() call on User::newSystemUser
* 305bc75b27 SessionManager: Don't generate user tokens when checking the tokens
* 7c4bd85d21 RequestContext::exportSession() should only export persisted session IDs
* 296ccfd4a9 SessionManager: Save 'persisted' flag in session metadata
* 94ba53f677 Move CSRF token handling into MediaWiki\Session\Session
* 46a565d6b0 Avoid false "added in both Session and $_SESSION" when value is null
* c00d0b5d94 Log backtrace for "User::loadFromSession called before the end of Setup.php"
* 4eeff5b559 Use $wgSecureCookie to decide whether to actually mark secure cookies as 'secure'
* 7491b52f70 Call session_cache_limiter() before starting a session
* 2c34aeea72 SessionManager: Abstract forceHTTPS cookie setting
* 9aa53627a5 Ignore auth cookies with value 'deleted'
* 43f904b51a SessionManager: Kill getPersistedSessionId()
* 50c5256352 SessionManager: Add SessionBackend::setProviderMetadata()
* f640d40315 SessionManager: Notify AuthPlugin when auto-creating accounts
* 70b05d1ac1 Add checks of $wgEnableBotPasswords in more places
* bfed32eb78 Do not raise a PHP warning when session write fails
* 722a7331ad Only check LoggedOut timestamp on the user loaded from session
* 4f5057b84b SessionManager: Change behavior of getSessionById()
* 66e82e614e Fix typo in [[MediaWiki:Botpasswords-editexisting/en]]
* f9fd9516d9 Add "bot passwords"
* d7716f1df0 Add missing argument for wfDebugLog
* a73c5b7395 Add SessionManager
Change-Id: I2389a8133e25ab929e9f27f41fa9a05df8147a50
Until Setup.php completes, we can't access the "current" user and trying
to do so logs a warning. So don't try.
Bug: T124367
Change-Id: If31230407829c04f34d9cfefcbb97edacb949b6d
The functionality for marking a debug log message for delivery to a log
file but not to MWDebug was removed in 1.25 with the introduction of
the PSR-3 logging system. Convert the messages that were marked for this
special handling in a7a0883 to use 'private' instead.
Bug: T122644
Change-Id: Iefaac07a5922c16c2899904c7b678199c5b1efe9
This overrides the message being sent. Use 'messageKey' instead,
which complements 'cacheKey' used elsewhere in the class.
Change-Id: I70b1115853d16e5f1357dc5f3d10e46f9c029fc6
MessageBlobStore class:
* Make logger aware.
* Log an error if json encoding fails.
* Stop using the DB table. WANObjectCache supports everything we need:
- Batch retrieval.
- Invalidate keys with wildcard selects or cascading check keys.
* Update tests slightly since the actual update now happens on-demand as
part of get() instead of within updateMessage().
ResourceLoader class:
* Remove all interaction with the msg_resource table. Remove db table later.
* Refactor code to use a hash of the blob instead of a timestamp.
Timestamps are unreliable and roll over too frequently for message blob store
because there is no authoritative source. The timestamps were inferred based on
when a change is observed. Message overrides from the local wiki have an
explicit update event when the page is edited. All other messages, such as
from MediaWiki core and extensions using LocalisationCache, have a single
timestamp for all messages which rolls over every time the cache is rebuilt.
A hash is deterministic, and won't cause needless invalidation (T102578).
* Remove redundant pre-fetching in makeModuleResponse().
This is already done by preloadModuleInfo() in respond().
* Don't bother storing and retreiving empty "{}" objects.
Instead, detect whether a module's message list is empty at runtime.
ResourceLoaderModule class:
* Make logger aware.
* Log if a module's message blob was not preloaded.
cleanupRemovedModules:
* Now that blobs have a TTL, there's no need to prune old entries.
Bug: T113092
Bug: T92357
Change-Id: Id8c26f41a82597e34013f95294cdc3971a4f52ae
Previously, logged-out users either only saw "Create account" and "Log in"
links in the personal area, or if $wgShowIPinHeader was true, they saw
[icon] 127.0.0.1 Talk for this IP address Create account Log in
where the IP address itself linked to the IP user page.
Now, logged-out users by default see the following:
[icon] Not logged in Talk Contributions Create account Log in
The old $wgShowIPinHeader feature is removed. It is very unfriendly to
show the user's IP address (in red, no less) at the top of every page,
since this will mean nothing to most visitors. Caching means that this
can't even be done reliably, anyway.
Another improvement is that the "talk" and "contributions" links are not
shown if anonymous users don't have the 'edit' right.
Modelled after the loggedOutTalkPage() function at Dutch Wikipedia
<https://nl.wikipedia.org/w/index.php?oldid=44706954>
Bug: T112724
Change-Id: I6f44e3e5d97ea917e4a03af47f3795792e4ca122
* Also added getBlob() and getBlobs() methods. These don't
require passing ResourceLoader as argument.
In preparation for T113092.
Change-Id: I3239c45f0243eca1eaf85bc53fca736e4c9a209c
The $ucfirst and $wgContentLang variables are not used by get().
Keep these inside getMessageFromFallbackChain() instead.
Also add mark getMsgFromNamespace() as public since it is used
in other places in core and by extensions.
Change-Id: I83bf77c17aa46f3929f89c4af97693611761629b
* Update and improve class documentation.
* Remove comment that claims to return false if the module has no message.
generateMessageBlob() returns '{}' in that case. And this is expected since
we want to cache the absence of messages.
* Re-use the module objects passed to MessageBlobStore::get() in getFromDB()
instead of using ResourceLoader::getModule(). The object is the same either
way (since ResourceLoader re-uses the objects also) but reduces complexity
of getFromDB() to not need the ResourceLoader object as parameter.
Change-Id: I89a14d7185877fae52791f6837883ed3a6749cd7
Language::ucfirst() already has this optimisation. The expensive
method is Language::uc(), which is only called when needed.
The ord() < 128 is one of the conditionals in Language::ucfirst().
Change-Id: Idd648b7b84eba1f92dda904438122fe38d3cf22b
Follows-up af89b09a06 and e40a90f0bc, and general phasing out
of MapCacheLRU and ProcessCacheLRU.
Use of ProcessCacheLRU seems redundant in this case as it is
essentially just a keyed singleton.
Change-Id: I7cc84cf6c76ecc422ea337dba2d216c3d7ebf281
In preparation for conversion from MapCacheLRU to HashBagOStuff.
* Use early-return pattern for error and rejection cases.
* Add 'int' to @return for getGoodLinkFieldObj() per the
values set in addGoodLinkObj().
* Add type hints for Title where possible.
* Avoid single-letter variables like $f and $s.
* Add visibility for public static methods.
Change-Id: Iab90539922f648c21cb147f749943a64f598e8de
* This avoids throwing atomic section errors when recache()
is triggered in the middle of transactions.
* Also made the read-only checks handle $wgReadOnly mode.
Change-Id: Ie00cf8454656a6dc3b1862475a959b2af3b472f0
This table is not needed because module names and their messages
array are available to the runtime environment at very little cost.
The only purpose it was serving is reverse lookup from message
key to module name (e.g. when MessageCache receives update that
need to propagate to MessageBlobStore). However that is better
achieved by simply looping through modules in PHP. The overhead
of a database is not worth this minor convenience.
MessageBlobStore
* insertMessageBlob: Doesn't need to update msg_resource_links.
* updateModule: Doesn't need to update msg_resource_links.
* getUpdatesForMessage: Reimplement with list from memory
instead of msg_resource_links.
The database table will be removed in If009e2620e59002e1.
Bug: T113092
Change-Id: Ia9131f570001f00c9800b260ac4b3469d54d2784
When the LocalisationCache class is set to 'detect', we prefer LCStoreCDB if
$wgCacheDirectory is set, but we default to LCStoreDB otherwise. Rather than
give up, we should try wfTempDir(), since any directory that meets its criteria
is also suitable for LCStoreCDB.
Change-Id: Id3e2d2b18ddb423647bf2e893bcf942722c0e097
Deprecated since MediaWiki 1.26 (32d1017e7d). These methods contain the
only remaining callers of MapCacheLRU::getAllKeys() which is the only odd
method of MapCacheLRU not supported by BagOStuff.
Change-Id: I64e054803595d49a5bd779accc94353924f7d87d
The $mForUpdate flag no longer causes the FOR UPDATE option to be added.
This was at first only true for some values of $wgAntiLockFlags (since
r9248 / ba8a00bcb833), though since 12757b50f8 (when the setting was
removed), is always the case.
The added text is based on the description for $wgAntiLockFlags.
Change-Id: I02a3e2df37ec40c7e36ae6210c8263f13d9e3e5b
CacheEpoch isn't meant for this (it's meant for page cache), I can't
imagine a scenario in which we'd want to bump that to invalidate
MessageBlobStore. It should be standalone and can be easily cleared
if needed by truncating the relevant table (it's automatically
repopulated).
This also removes wfTimestamp/DateTime overhead.
Change-Id: Iab06edbf71f20f3430207a80df90131c79dc03a7
These callers don't need to do purges, but can still perfectly
take advantage of this instance over a plain BagOStuff. Namely:
* Replication and snapshot lag awareness
* Preemptive regeneration
* Easy process cache support
The idea is for there to only be one caching class/factory
to use, instead of having rules for picking which one to use.
Change-Id: I8e362df451c0c28731fc853c044c4c4b8e097f01
LocalisationCache was added in 1.16, so in order to retain PHP 5.1
compatibility, array_fill_keys() was avoided. In 1.17, support for
PHP 5.1 was dropped, so we can use that function now.
Change-Id: I435705639f1a470324a4ba46153351aadc0d40e2
* If replace() was called recently, then we know that this is the
master data center and that the messages are up-to-date. With this
change, replace() calls have nothing to contend with aside from other
replace() calls. Even if there were timeouts due to such contention,
caused by high MediaWiki: page edit rates, the replace() updates would
pick up the prior changes in passing since they do load().
* This also avoids the following scenario:
a) Someone edits a message page, triggering replace()
b) Some page view causes load() to trigger loadFromDB()
due to the hash being seen as volatile due to replace()
c) The loadFromDB() loads stale slave data and undoes
the message key update the replace() did
Change-Id: I9cdf7ad3d67f168fcba7f633af9e32a8d1fa928e
* Made the status key only act as a backoff key inside
loadFromDBWithLock(), instead of also being a lock key. The
getReentrantScopedLock() call is now non-blocking, which has
a similar affect to the add() call on the status key but much
better prioritizes replace(); that method already has a blocking
getReentrantScopedLock() call, so once it gets that lock, there
is no worry of load() failing due to contention.
* Bail out in loadFromDBWithLock() if the scope lock was not acquired.
Normally the status lock often assured the lock could be obtained,
unless a competing replace() came in. The old status lock would
cause a bail if not acquired, and this carries over that behavior
but also avoids clobbering updates when replace() contention happens.
* Avoid calling saveToCaches() in replace() if the lock was not
acquired to avoid clobbering conflicting writes.
* Made the loadFromDB() call in replace() use DB_MASTER to avoid
seeing stale data if the cache is volatile.
* Lowered WAIT_SEC, as the default PHP timeout is 30.
We want this to be able to at least finish the other calls
in replace() even if the lock times out.
* Avoid checking the status key an extra time in load().
* Made a few small code and doc cleanups.
Change-Id: Ibaf1f67618ec374c83c3135a71e90223dd2b1856
* The key salting of the invidial keys via the WAN cache makes
it OK to use slaves here. The salting happens via the delete()
in the MessageCache::replace() call, which is hit when the
message page is edited.
Bug: T92357
Change-Id: Ic191f6c3fc49c4c58461d26468dd0fa94a52051e
* These keys are used for larger messages not in the main blob,
and they still use explicit purges as the pages change
Change-Id: I254ddcc9ffe7c49654788d6aabcca2ab7ed4f03f
* The localCache member will use CACHE_NONE if this is off,
so the set()/get() calls are already no-oped
* Also allow using the local cache if $hash is false
in "stale mode", which handles the case where the hash
key fell out of memory for some reason more gracefully
Change-Id: Ie12efcd4088a6dc4a4cdd2fd06646f2881df53d7
* This avoids constant churn when $wgUseLocalMessageCache is set to false
* Follow up to db464b8a84
Bug: T109183
Change-Id: I8da324c53527da32d09964be6c3a92176af4ee7b
Follow-up for I020617d, where I got this wrong. It is probable that multiple
application servers will try to build a local cache at the same time, in which
case they will cause the cache key to thrash. It is necessary that two
application servers building a local cache from the database generate the same
hash for a given set of messages.
Change-Id: Ieeefc2094a83be9401c466bec859c1588ddfbcdf
* Use hash key volatility to propagate invalidations over DCs
in addition to memcached->APC instances.
* Make use of stale APC cache due to hash mismatch instead
of waiting around in some cases. This is similar to the
TTL expiry and volatility cases.
* Renamed $hashExpired and added some comments for clarity.
Change-Id: I8890fb174cae72c4ce8872df64f52f5cbd55183b
In addition to eliminating disk IO in a hot path, using APC spares us from
having to serialize and unserialize cache arrays. Since we're not serializing,
though, we don't have a string representation to hash, so use a random string
instead. (The code already treats the association of hash string to cache as
purely symbolic, so this is not problematic.)
Whereas the hash was previously stored as the first 32 bytes of each cache
file, we now store it as an array key instead (like VERSION and EXPIRY were
already). Because this changes the structure of cached data, we have to bump
MSG_CACHE_VERSION.
While we're here, make MessageCache::getLocalCache() and
MessageCache::saveToLocal() protected, make their signatures more
consistent with other methods in this class. While they were (implicitly)
public before, there are absolutely no external callers in Core or
extensions[0][1], so we can skip the standard deprecation process.
[0]: https://github.com/search?q=%40wikimedia+getlocalcache&type=Code&utf8=%E2%9C%93
[1]: https://github.com/search?utf8=%E2%9C%93&q=%40wikimedia+savetolocal&type=Code
Change-Id: I020617d2df2a8f0f243b85f3383dc7b16f15aaad
Especially in long running maintenance scripts, this can be problematic.
LinkCache is now LRU-based, and will store a maximum of 10,000 good titles,
and 10,000 bad ones.
LinkCache::getGoodLinks() and getBadLinks() are deprecated since they
problematic to support in this implementation and are unused.
Bug: T106998
Change-Id: I1328149d65a5e75a5d6e10cb2686a099562a1847
Missing global declaration when using $wgContLang in
MessageCache::normalizeKey
Change-Id: Ia1c1f41244fd5629527b99a5f2038f607cff42c4
Follow-up: 47e0f0c3
* Normalize the message name returned by allmessages
* Separate message key normalization into
MessageCache::normalizeKey()
Bug: T63894
Change-Id: I1d89fc73fea705243d390bab91255a635d8f9eee
The variable $row is set to the last user of the first loop.
It is better to use the $name which is set in the foreach correctly.
This now adds all userpages to the LinkBatch and that avoids some extra
queries on at least Special:ListFiles.
Change-Id: Ied378b1596ec9d38eda41ce5ee413203c65eb21b
strtr() is marginally faster as it runs through the string only
once. A better fit for one-for-one character translation.
The strtr() function also supports an associative array as second
parameter for entire string replacements. This, too, has the same
performance and predictable behaviour (starts with the longest key).
Whereas str_replace is for more aggressive needs where you want
multiple passes until there are no further matches.
The associative array form is arguably also easier to understand
and harder to mess up since the needle/replacement pairs are
explicitly connected instead of two separate arrays.
Also:
* Use getFormattedNsText instead of strtr( getNsText, .. ) which
reduces duplication of this fact through a more semantic intent.
Change-Id: Ie23e4210a5b6908dd79eebc8a2b931d12fe31af6
Since MediaWiki 1.24 a constructor is defined in HTMLFileCache,
so the constructor of the parent class, FileCacheBase, is no more
called, and $wgUseGzip is no more used. This fix explicitely calls
the parent constructor.
Bug: T103237
Change-Id: I80c1871881049b9618c23aa76e6665867ecec2aa
LinkBatch::add already handle the underscore/space part, that means it
is not need to do it on the caller side when adding user names to
LinkBatch
Change-Id: I09e80712903a539164141cc0a88d321203114677
- Removed space after casts
- Removed spaces in array index
- Added spaces around string concat
- Added space after words: switch, foreach
- else if -> elseif
- Removed parentheses around require_once, because it is not a function
- Added newline at end of file
- Removed double spaces
- Added spaces around operations
- Removed repeated newlines
Bug: T102609
Change-Id: Ib860222b24f8ad8e9062cd4dc42ec88dc63fb49e
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.
Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.
Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.
Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
Implementation written by Fred Emmott of Facebook. Quoting Fred:
As well as array access being faster, the main advantage is actually
that this significantly reduces the use of unserialize(), which does a
lot of memcpys when making the strings.
Benchmarks compared to LCStoreCDB:
* HHVM (no repo-auth): ~7% improvement
* HHVM (with repo-auth): ~12% improvement
* PHP7: ~1% improvement
My (Legoktm) brief testing noted that the generated PHP files were
noticiably larger than the CDB ones:
* 1.5M en.l10n.php
* 932K l10n_cache-en.cdb
Bug: T99740
Change-Id: Ib2c5856d40cd928cab4a79cb935b3ce08c598300
* This should not be used, and load() does not see them.
If the content language is 'de', then message overrides
will not include MediaWiki:<key>/de pages.
Change-Id: Ie4b6b356bd309814dd4a88040a29a7ebd509712a
* The cache has to reload and *after* locking to avoid
losing any concurrent changes.
* Also fixed incorrect assumption in MessageCacheTest.
Message overrides for the content language do not use
the language suffix.
Change-Id: I98ff158a1575330bc59efe6badb27f8de8717951
This allows us to continue to use MessageBlobStore::get() as a way
to pre-fetch messages in batches for multiple modules while also
allowing individual modules to retrieve their messages in a convenient
way.
This is in preparation for implementing ResourceLoaderModule::buildContents
and using it in ResourceLoaderModule::getVersionHash.
ResourceLoader::respond() already fetches message blobs in batches for the
actual response, but it also needs getVersionHash(), which would otherwise
fetch the same messages a second time.
Change-Id: I7e4c8b65765b54807123e85cfbb7eb2e5b2f39bd
Modules now track their version via getVersionHash() instead of getModifiedTime().
== Background ==
While some resources have observeable timestamps (e.g. files stored on disk),
many other resources do not. E.g. config variables, and module definitions.
For static file modules, one can e.g. revert one of more files in a module to a
previous version and not affect the max timestamp.
Wiki modules include pages only if they exist. The user module supports common.js
and skin.js. By default neither exists. If a user has both, and then the
less-recently modified one is deleted, the max-timestamp remains unchanged.
For client-side caching, batch requests use "Math.max" on the relevant timestamps.
Again, if a module changes but another module is more recent (e.g. out-of-order
deployment, or out-of-order discovery), the change would not result in a cache miss.
More scenarios can be found in the associated Phabricator tasks.
== Version hash ==
Previously we virtually mapped these variables to a timestamp by storing the current
time alongside a hash of the value in ObjectCache. Considering the number of
possible request contexts (wikis * modules * users * skins * languages) this doesn't
work well. It results in needless cache invalidation when the first time observation
is purged due to LRU algorithms. It also has other minor bugs leading to fewer
cache hits.
All modules automatically get the benefits of version hashing with this change.
The old getDefinitionMtime() and getHashMtime() have been replaced with dummies
that return 1. These functions are often called from getModifiedTime() in subclasses.
For backward-compatibility, their respective values (definition summary and hash)
are now included in getVersionHash directly.
As examples, the following modules have been updated to use getVersionHash directly.
Other modules still work fine and can be updated later.
* ResourceLoaderFileModule
* ResourceLoaderEditToolbarModule
* ResourceLoaderStartUpModule
* ResourceLoaderWikiModule
The presence of hashes in place of timestamps increases the startup module size on
a default MediaWiki install from 4.4k to 5.8k (after gzip and minification).
== ETag ==
Since timestamps are no longer tracked, we need a different way to implement caching
for cache proxies (e.g. Varnish) and web browsers. Previously we used the
Last-Modified header (in combination with Cache-Control and Expires).
Instead of Last-Modified (and If-Modified-Since), we use ETag (and If-None-Match).
Entity tags (new in HTTP/1.1) are much stricter than Last-Modified by default.
They instruct browsers to allow usage of partial Range requests. Since our responses
are dynamically generated, we need to use the Weak version of ETag.
While this sounds bad, it's no different than Last-Modified. As reassured by
RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.3> the
specified behaviour behind Last-Modified follows the same "Weak" caching logic as
Entity tags. It's just that entity tags are capable of a stricter mode (whereas
Last-Modified is inherently weak).
== File cache ==
If $wgUseFileCache is enabled, ResourceLoader uses ResourceFileCache to cache
load.php responses. While the blind TTL handling (during the allowed expiry period)
is still maxage/timestamp based, tryRespondNotModified() now requires the caller to
know the expected ETag.
For this to work, the FileCache handling had to be moved from the top of
ResoureLoader::respond() to after the expected ETag is computed.
This also allows us to remove the duplicate tryRespondNotModified() handling since
that's is already handled by ResourceLoader::respond() meanwhile.
== Misc ==
* Remove redundant modifiedTime cache in ResourceLoaderFileModule.
* Change bugzilla references to Phabricator.
* Centralised inclusion of wgCacheEpoch using getDefinitionSummary. Previously this
logic was duplicated in each place the modified timestamp was used.
* It's easy to forget calling the parent class in getDefinitionSummary().
Previously this method only tracked 'class' by default. As such, various
extensions hardcoded that one value instead of calling the parent and extending
the array. To better prevent this in the future, getVersionHash() now asserts
that the '_cacheEpoch' property made it through.
* tests: Don't use getDefinitionSummary() as an API.
Fix ResourceLoaderWikiModuleTest to call getPages properly.
* In tests, the default timestamp used to be 1388534400000 (which is the unix time
of 20140101000000; the unit tests' CacheEpoch). The new version hash of these
modules is "XyCC+PSK", which is the base64 encoded prefix of the SHA1 digest of:
'{"_class":"ResourceLoaderTestModule","_cacheEpoch":"20140101000000"}'
* Add sha1.js library for client-side hash generation.
Compared various different implementations for code size (after minfication/gzip),
and speed (when used for short hexidecimal strings).
https://jsperf.com/sha1-implementations
- CryptoJS <https://code.google.com/p/crypto-js/#SHA-1> (min+gzip: 2.5k)
http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js
Chrome: 45k, Firefox: 89k, Safari: 92k
- jsSHA <https://github.com/Caligatio/jsSHA>
https://github.com/Caligatio/jsSHA/blob/3c1d4f2e/src/sha1.js (min+gzip: 1.8k)
Chrome: 65k, Firefox: 53k, Safari: 69k
- phpjs-sha1 <https://github.com/kvz/phpjs> (RL min+gzip: 0.8k)
https://github.com/kvz/phpjs/blob/1eaab15d/functions/strings/sha1.js
Chrome: 200k, Firefox: 280k, Safari: 78k
Modern browsers implement the HTML5 Crypto API. However, this API is asynchronous,
only enabled when on HTTPS in Chromium, and is quite low-level. It requires boilerplate
code to actually use with TextEncoder, ArrayBuffer and Uint32Array. Due this being
needed in the module loader, we'd have to load the fallback regardless. Considering
this is not used in a critical path for performance, it's not worth shipping two
implementations for this optimisation.
May also resolve:
* T44094
* T90411
* T94810
Bug: T94074
Change-Id: Ibb292d2416839327d1807a66c78fd96dac0637d0
Seriously, the ops team spent some time trying to find that page during an outage,
while in fact it's an obscure Tolkien reference - better be clear.
Also, set the other dummy titles to something very clearly explaining what's
going on and where.
Change-Id: I6f33a2ea5030f22a258830a33f7bcefa7f0acd85
* This ends up being more complex than its worth
and even more so for multi-DC support
Bug: T93006
Change-Id: Iaa774fe69061e42955b11dc82d30dba93208e606
The test for OutputPage::makeResourceLoaderLink was triggering database
queries through MessageBlobStore even though it doesn't use any
messages.
In bb03d1a8e0 I had made MessageBlobStore a singleton instead of static
functions, however there's no need for it to be one since the class is
stateless. Callers can just create a new MessageBlobStore instance and
call functions upon it. Using getInstance() is now deprecated.
ResourceLoader now has a setMessageBlobStore setter to allow overriding
which MessageBlobStore instance will be used. OutputPageTest uses this
to set a NullMessageBlobStore, which makes no database queries.
Change-Id: Ica7436fb6f1ea59bd445b02527829ab0742c0842
We review documentation all the time. Even if this was a big, notable
review, it was 5 years ago. It's probably outdated again, e.g. because
methods changed but the corresponting documentaion did not. In my
opinion the fact that a review happened 5 years ago is not useful any
more.
Change-Id: I6f4fb88ea790520bf2443aae4144cdde394b5e78
* Use special prioritized refreshLinksJobs instead, which triggers when
transcluded pages are changed
* Also added a triggerOpportunisticLinksUpdate() method to handle
dynamic transcludes
bug: T89389
Change-Id: Iea952d4d2e660b7957eafb5f73fc87fab347dbe7
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.
Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.
Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
DOMDocument::load fails to load the plurals data during the import
process.
This is a work-around for https://bugs.php.net/bug.php?id=64938. This
bug only rears its head when using Special:Import, because that is
essentially the only place in MediaWiki where we fiddle with
libxml_disable_entity_loader.
Bug: T58439
Change-Id: Idcb4ab1cef2a7b080543e7cc1cee5464fc476456
ResourceLoaderImageModule needs a set of SVG files and some data in
the module definition, and produces styles for a set of CSS classes,
one for each image, optionally with differently colored variants,
generated in SVG and PNG, data-URI-embedded if possible, compatible
with all browsers, and generally slick.
The intended usage is to ship icon libraries with MediaWiki that can
be used throughout the pages with no additional code.
* ResourceLoaderImageModule implements all of the logic for data
parsing and CSS generation.
* ResourceLoaderImage implements the logic for SVG image colorization
(for variants) and rasterization.
* ResourceLoader and ResourceLoaderContext were extended to serve a
new kind of load.php request that delivers a single image file. This
is used for fallback PNG images served to browsers that don't
understand SVG.
See change Ic6a76bfb for a demo.
Bug: T76473
Co-Authored-By: Trevor Parscal <trevorparscal@gmail.com>
Co-Authored-By: Bartosz Dziewoński <matma.rex@gmail.com>
Change-Id: Idf6ff4eb8e94f45946f15d283d34108b881fae6e
- Added/removed spaces around parenthesis
- Added newline in empty blocks
- Added space after switch/foreach/function
- Use tabs at begin of line
- Add newline at end of file
Change-Id: I244cdb2c333489e1020931bf4ac5266a87439f0d
If $wgMessagesDirs is initially empty, we can optimize when batch loading
extensions:
if ( !$wgMessagesDirs ) {
$wgMessagesDirs = $cache['MessagesDirs'];
}
With APC, this should be O(1) CPU time.
This was suggested by Tim in the code review of I7074b65d07c5.
Change-Id: I66fa907cdaafe18b74b5b9afaa8b6b1db069bea3
The new cdb library is pulled in via composer. Since the
library uses namespaces, a backwards-compatability layer
is provided for the old class names:
* CdbReader
* CdbWriter
* CdbException
The PHP/DBA-specific classes should never have been used directly.
Depends on I98302bdf1 in mediawiki/vendor
Change-Id: I39549ac8540b262cf91f7d1830d36327afb3033d
It's a very hot code path, so it makes sense to be able to dive into it in
detail, and having a discrete log bucket makes that easier to do.
Change-Id: I4b547e97711b9e45a99f7b747d785690c70883ec
- Added newline at end of file
- Removed double spaces/newlines
- Added space after if/function and parentheses/brackets
- Removed space before comma/cast
- Fixed indent of some lines
Change-Id: I29867ffdffdfb7d2b56997e9393497c7dc12f7d3
Needed for MobileFrontend on third-party sites that use this feature.
Ideally, there should be a way to cache mobile requests too, but that's
a different issue requiring much more effort to do properly.
Bug: 68106
Change-Id: I01a76c571d9186b325f19a00cec136459707c791
* Since individual message keys can expire, this can cause broken
messages when such a key is needed (the message is treated as if
it did not exist). The base class clearly documents a need for
more atomicity (only top level keys can fall it separately).
Change-Id: I992bba77a0afdeeeade8be013708277b79f22314
Currently LocalisationCache merges the core data for all languages in
the fallback chain, then the extension data, then merges those two, and
then gives extensions like LocalisationUpdate a chance to make final
overrides with the LocalisationCacheRecache hook.
But if LocalisationUpdate doesn't want to locally duplicate all the
messages for every language (e.g. r104041), LocalisationCacheRecache is
too late: the information as to whether a message came from the primary
language or a fallback has been lost, so when LU itself has an override
for a fallback language it can't know whether or not the existing
message should be overridden or not.
The solution is for LocalisationCache to gather the data for each
fallback language separately, call a new hook for LU to affect just that
language (LocalisationCacheRecacheFallback), and only then merge the
fallback languages together.
Bug: 68781
Change-Id: Iacfe96063fcc66c1f97ca5e5292a8fc70af988cf
* Implemented a version of BloomCache using Redis
* Added a BloomCheckTitleHasLogs handler class for avoiding
slow logging table queries when large amounts of 404 pages
are viewed (by various web crawlers at the moment).
bug: 67439
Change-Id: I26e5034755e3a7208a45991b1cf2f12467679cc1
For easier testability and other things. There are no uses
of this class in any extensions in gerrit.
Change-Id: I606de4259239e128ed7e0477fc98b84c647430c4
* Previously the cache size was unbounded and leaky
* Also made MapCacheLRU handle null values properly
Change-Id: Ia02258cf051e1ccf11457c758741b8c7922afe89
Also don't cast $model to int in LinkCache::addGoodLinkObj(); content
model IDs are non-numeric strings, not integers, so that field was
always populated with the value 0. Because 0 is a falsy value, this
caused subsequent calls to Title::getContentModel() to return the
default model rather than the correct one.
Also (hopefully) fixed every single query that could cause a
LinkCache entry to be added without the content model.
Bug: 69789
Change-Id: I94f06baf406afa538cd2b10139598442f9fc6759
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling
Change-Id: I7b65fe04db431342cc58b469dc48f41a50c4e891
* This avoids breaking the main transaction in finishWrite().
* Also removed an unused variable in SqlBagOStuff.
Change-Id: Ia330ac362b080c1338616d95b793032c4752dc23
* Only do the DB inserts in finishWrite(). Previously this would
COMMIT there but would spread out queries throughout the set()
calls. This pushes all the contention down to the last step.
Change-Id: Ia8afb79a8328c18a7d8a386bcd15a5074007d52e
Change-Id I427c6de5a0a29b43cff755db0eb8a750db620173 increases the
probability that a null byte will attempt to be stored in the
lc_value column. PostgreSQL does not allow that byte in a text
column, so convert the column to bytea.
If the column already contains corrupted data, the upgrade routine
might fail. To prevent this, delete the contents of the table before
changing the type.
Bug: 62098
Change-Id: Ie8368bde398b2ae4d3cfc9ee7bf35874bd2ded68
These files have all had treatment before, and these occurrences have either
been missed or have been introduced after.
Change-Id: I06cdab4616b5bff47c85152df28f18c861730a23
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.
Change-Id: I86fd10e3f2d4bb80e7432533038d124693acfb3c
For now, non-message data (e.g. special page aliases, magic words) belong
in separate files having their own keys in $wgExtensionMessagesFiles. It
is unnecessary to read any PHP files listed under keys that exist in
$wgMessagesDirs; they are merely compatibility shims.
Bug: 63926
Change-Id: I92013d0a45e83ad0f5ac483a3db867806eb71f8a
Statistics are now showing always no messages since languages.inc
does not manage to load the JSON messages.
Had to make LocalisationCache::readJSONFile() public so that it can
be used in languages.inc; since all methods from LocalisationCache
return localisations merged with fallbacks, which is not what we
want here.
Fix for I918cfdc46c (0dd91d5).
Change-Id: Ib52287db618b9d072e847130070d165a3e7ae44b
LocalisationCache and Language have to take the JSON files into account
in deciding if a language is present or not.
Standardizing language validity checking with isSupportedLanguage
and isValidBuiltInCode.
Co-Authored-By: Niklas Laxström <niklas.laxstrom@gmail.com>
Co-Authored-By: Siebrand Mazeland <siebrand@kitano.nl>
Change-Id: I35bbb3a7a145fc48d14fff620407dff5ecfdd4fc
Rationale is the same as for Ifb6dc2666f (96b04ce): ability to
separate it form the main log group, as it always has a predicitible
value.
Change-Id: I0ac5baf5ab8fbe4d2025ccf6439b883dc82e9d5b
Also removed true as second parameter to it from CloneDatabase.php
since it is the default value of that parameter.
Change-Id: I727ebae2bd4df0e26019985ce8c7ce73381c5642
Example usage:
$wgHooks['MessageCache::get'][] = function( &$key ) {
static $keys = null;
if ( $keys === null ) {
global $wgExtensionMessagesFiles;
require( $wgExtensionMessagesFiles['OverrideMessages'] );
$keys = array_flip( array_keys( $messages['en'] ) );
}
if ( isset( $keys["myprefix-$key"] ) ) {
$key = "myprefix-$key";
}
return true;
}
Pros:
* Easy way to override standard core and extension messages without
any changes to them
* Messages can be stored in a standard i18n file
* Messages can be translated easily with Translate
* Messages can be shared accross multiple wikis easily
* Takes advantage of the normal message cache behavior unlike the
MessagePreLoad hook
* Missing translations fallback to the override, not to the
uncustomized standard translation
* Do not need to handle conflicting message keys at translatewiki.net
if adopted by WMF
Cons:
* This method is called often, so there will be small performance
impact if no hooks are registered. Impact can be big if the
implementation of hook subscriber is inefficient.
This can help with bugs like 36149. It doesn't remove the manual work
needed to detect those messages and adding them to the i18n file.
I have been using this patch in a wiki farm for months.
Change-Id: Ib39937a440e71ae7292cf992ab37a569189741e4
- The parameter is now a string, making is more understandable than
boolean values
- It takes the same values in both wfDebug() and wfDebugLog() (except
for 'private' which is only used in the latter)
- This adds a new possibility to wfDebugLog() to log the message either
on the specific log or the general one, but not to the debug toolbar
- Old boolean values are still recognised for backward compatibility
- Also send the messages passed to wfDebugLog() to the debug toolbar
when they are written to a specific log and not restricted to logs
- Updated the calls of and wfDebug() and wfDebugLog() with the last
parameter to change it into a string
- Renamed MWDebug::sendWarning() to MWDebug::sendMessage() and added
$group parameter to it; will not break anything since that method
is marked as private
- Changed the call to wfDebug() from MWDebug::sendMessage() to use
wfDebugLog() with 'log' as thrid parameter, so that those messages
can be logged separately from the main log and they don't show up
a second time on the "debug log" tab of the debug toolbar
Change-Id: I1be09d4c1d3408ed5b26a5db02691c17c0ec0926
Added wfSuppressWarnings and wfRestoreWarnings before and after the
ini_set call to apc cache by default to suppress errors if ini_set is
disabled.
Change-Id: If91e99c30fce3af14e69acd35112af188e62b6ac
Implementation for https://www.mediawiki.org/wiki/Requests_for_comment/Localisation_format
Add $wgExtensionMessagesDirs, which tracks the directory
(or directories) where each extension stores it's JSON i18n files.
In this commit only support for messages is implemented, but adding
support for other i18n variables (e.g. magic words) is easy to do later.
To be backwards compatible, an extension can specify both
$wgExtensionMessagesFiles and $wgExtensionMessagesDirs. Older versions
of MediaWiki will just work, and newer versions will use the JSON files
while ignoring the PHP file (except if the PHP file contains non-message
data like magic words).
Misc changes:
* Updated mergeMessageFileList.php to output both
$wgExtensionMessagesFiles and $wgExtensionMessagesDirs
Change-Id: I8d137e15e1670880a9847263e6ce796c62a4670d
* New operands i, v, w, f, t
* New operators =, !=, %
* Ignore "samples", which are basically unit tests embedded in rule
specifications
* Ignore the new "other" rules, which have an empty condition. It
doesn't really makes sense to parse them, since the empty condition
means special handling should be done in the caller, it is not
equivalent to an unconditional true or false.
* Trailing zero support requires that the input number be a string.
Documented this.
* Fixed some comments
* Added test cases for new features
Bug: 56931
Change-Id: I96986c0c664f785e75b0a4ced2ec9e37b72681c1
