Commit graph

174 commits

Author SHA1 Message Date
jenkins-bot
d69b967481 Merge "MessageCache: replace should actually replace, not reload" 2018-10-11 20:56:33 +00:00
jenkins-bot
7c091e68e9 Merge "messagecache: avoid caching message pages that do not override" 2018-10-11 18:59:24 +00:00
Brad Jorsch
08e9eb1fef MessageCache: replace should actually replace, not reload
Prior to I462554b30, MessageCache::replace() did just that: it took the
existing cache and updated the one entry.

In I462554b30, the rearrangement of work into a DeferredUpdate
introduced a bug: the in-process cache was updated, but when the shared
cache was loaded later the entry was never updated in there so the
shared caches kept the old value. This was found in code review and
worked around by reloading all the messages from the database instead of
updating the existing cache.

But all that extra work reloading everything from the database causes
major slowness saving any MediaWiki-namespace page when the wiki has
many such small pages. Let's go back and fix the bug so replace() can
again replace instead of reloading everything.

Bug: T203925
Change-Id: Ife8e1bd6f143f480eb8da09b817c85aadf33a923
2018-10-11 11:20:51 -04:00
Aaron Schulz
a3d6c1411d messagecache: avoid caching message pages that do not override
Clean up individual message cache handling when the master key is
volatile. The goal is not to treat the message as gone but to re-fetch
it when accessed rather than cache the value in the main/process cache.

Bug: T193271
Change-Id: I4bcaf10c1516e7c96c2d0963722affeaf80272e0
2018-10-10 12:35:07 -07:00
Aaron Schulz
59242afaa7 messagecache: use MergeableUpdate for the deferred replace() update
This combines the load loop for multiple messages for a language code.

Bug: T203925
Bug: T193271
Change-Id: Ie5e1e83d6740344b7ca641c99fb3bd4ad5718492
2018-10-05 20:14:41 +00:00
Aaron Schulz
548c92a332 MessageCache: do not store the EXCESSIVE array as it is only needed for HASH
This saves space as the number of entries becomes larger.

Bug: T193271
Change-Id: I32d7512d7f05ae7c9a69b88cd19df64d7a21b4a1
2018-10-04 02:22:05 +00:00
Aaron Schulz
7d2b121ac8 MessageCache: remove confusing and unused $isFullKey parameter from get()
Follows-up cba0fb1c15, which removed the last caller.

Change-Id: I00c17fedff39b1b35519cff2a0f8eac3e4d6f2ab
2018-10-02 23:34:53 +00:00
Umherirrender
173df1768b Fix logged method in MessageCache::replace/loadCachedMessagePageEntry
Seeing {closure} in the logs as caller is not helpful

Change-Id: Iddfb6042bba2a59a9f21f533b748c96478a87b97
2018-09-30 17:37:51 +00:00
Umherirrender
40d3ce8d17 Reduce calls to MediaWikiServices::getInstance()
In some functions MediaWikiServices::getInstance() was called twices or
in loops. Extract the variable to reduce calls.

Change-Id: I2705db11d7a9ea73efb9b5a5c40747ab0b3ea36f
2018-08-18 06:02:39 +02:00
jenkins-bot
a8b828a774 Merge "Avoid MapCacheLRU error when MessageCache fails to load" 2018-08-14 16:32:07 +00:00
Fomafix
0a0d5cb7f7 Fix typos
Bug: T201491
Change-Id: I25a27d11faabe2f5fa02950c7a4fb58b13fb3662
2018-08-14 09:52:19 +00:00
Aaron Schulz
65ad02955c Avoid MapCacheLRU error when MessageCache fails to load
Bug: T201893
Change-Id: I6093113a3ffa8092dea3351a6ed6c815c7ff7162
2018-08-13 23:03:27 -07:00
Aryeh Gregor
90d4f56fe4 Mass conversion of $wgContLang to service
Brought to you by vim macros.

Bug: T200246
Change-Id: I79e919f4553e3bd3eb714073fed7a43051b4fb2a
2018-08-11 22:44:29 -06:00
Aaron Schulz
70b38d4d2b Make MessageCache use APC for big messages
Bug: T118893
Change-Id: I418d0ca490911d7ee4213fe1728cfbc637ef8b10
2018-07-25 13:24:52 -07:00
Timo Tijhof
c9bcbaa558 cache: Minor docs for MessageCache interaction with WANObjectCache
Follows-up 04bc03a29a.

Change-Id: I8fb810fc2e08277bb64fa18595483a2161cfb1cc
2018-07-25 13:28:19 +00:00
Aaron Schulz
04bc03a29a Make MessageCache use getWithSetCallback() for big messages
Change-Id: I8f93e1dffbcbbabf4d7cb361b001d8ca6e7ad954
2018-07-19 12:41:14 +00:00
Umherirrender
130ec2523d Fix PhanTypeMismatchDeclaredParam
Auto fix MediaWiki.Commenting.FunctionComment.DefaultNullTypeParam sniff

Change-Id: I865323fd0295aabd06f3e3c75e0e5043fb31069e
2018-07-07 00:34:30 +00:00
Aaron Schulz
97e86d934b Limit the number of cached languages in MessageCache via MapCacheLRU
Change-Id: I37a128cfc553e0edaab524098461776cec3fe08a
2018-07-03 11:03:14 +00:00
Aaron Schulz
5c0937b167 Make MessageCache::getMsgFromNamespace() process cache hook-defined keys
Change-Id: I6f8a0c002b54dc48bc3e6902ab1ba41b2f99412a
2018-06-23 17:59:12 +01:00
Bartosz Dziewoński
0313128b10 Use PHP 7 "\u{NNNN}" Unicode codepoint escapes in string literals
In cases where we're operating on text data (and not binary data),
use e.g. "\u{00A0}" to refer directly to the Unicode character
'NO-BREAK SPACE' instead of "\xc2\xa0" to specify the bytes C2h A0h
(which correspond to the UTF-8 encoding of that character). This
makes it easier to look up those mysterious sequences, as not all
are as recognizable as the no-break space.

This is not enforced by PHP, but I think we should write those in
uppercase and zero-padded to at least four characters, like the
Unicode standard does.

Note that not all "\xNN" escapes can be automatically replaced:
* We can't use Unicode escapes for binary data that is not UTF-8
  (e.g. in code converting from legacy encodings or testing the
  handling of invalid UTF-8 byte sequences).
* '\xNN' escapes in regular expressions in single-quoted strings
  are actually handled by PCRE and have to be dealt with carefully
  (those regexps should probably be changed to use the /u modifier).
* "\xNN" referring to ASCII characters ("\x7F" and lower) should
  probably be left as-is.

The replacements in this commit were done semi-manually by piping
the existing "\xNN" escapes through the following terrible Ruby
script I devised:

  chars = eval('"' + ARGV[0] + '"').force_encoding('utf-8')
  puts chars.split('').map{|char|
    '\\u{' + char.ord.to_s(16).upcase.rjust(4, '0') + '}'
  }.join('')

Change-Id: Idc3dee3a7fb5ebfaef395754d8859b18f1f8769a
2018-06-04 16:20:13 +00:00
Bartosz Dziewoński
485f66f174 Use PHP 7 '??' operator instead of '?:' with 'isset()' where convenient
Find: /isset\(\s*([^()]+?)\s*\)\s*\?\s*\1\s*:\s*/
Replace with: '\1 ?? '

(Everywhere except includes/PHPVersionCheck.php)
(Then, manually fix some line length and indentation issues)

Then manually reviewed the replacements for cases where confusing
operator precedence would result in incorrect results
(fixing those in I478db046a1cc162c6767003ce45c9b56270f3372).

Change-Id: I33b421c8cb11cdd4ce896488c9ff5313f03a38cf
2018-05-30 18:06:13 -07:00
jenkins-bot
c122357c72 Merge "Switch the sidebar cache to using checkKeys" 2018-02-20 22:09:50 +00:00
Aaron Schulz
cf54d68968 Switch the sidebar cache to using checkKeys
The avoids the long delete() loop in MessageCache::replace()
and has better separation of concern.

Change-Id: I0acb0119058fa92fcafb52a5850f5dad4aaa94d2
2018-02-19 04:10:39 -08:00
Aaron Schulz
394b272a64 Cleanup IDEA warnings in MessageCache
Change-Id: I0ced7c7289918f78bb6f3682ae974fe0c04a1e04
2018-02-19 12:08:15 +00:00
Brad Jorsch
2791fb0861 Hard-deprecate ParserOutput stateful transform methods
This also removes all the in-core calls that had been kept for the
benefit of extensions, and causes them to not have any effect since
anything that had been calling them was already either a no-op or will
probably be broken now that nothing in core is setting or checking the
flags.

Change-Id: Id22c1a5a6d6a249debb14063ae3f8838d105b634
2018-02-13 12:28:36 -05:00
Brad Jorsch
d511626236
Add 'unwrap' ParserOutput post-cache transform
And deprecate passing false for ParserOptions::setWrapOutputClass().

There are three cases for the Parser wrapper: the default
mw-parser-output, a custom wrapper, or no wrapper. As things currently
stand, we have to fragment the parser cache on each of these options,
which uses a nontrival amount of storage space (T167784).

Ideally we'd do all the wrapping as a post-cache transform, but
TemplateStyles needs to know the wrapper in use in order to properly
prefix its CSS rules (that's why we added the wrapper in the first
place). So, second best option is to make *un*wrapping be a post-cache
transform and make "custom wrapper" be uncacheable.

This patch does the first bit (unwrapping as a post-cache transform),
and a followup will do the second part once the deprecation process is
satisfied.

Bug: T181846
Change-Id: Iba16e78c41be992467101e7d83e9c3134765b101
2018-02-01 14:24:27 -08:00
Umherirrender
3124a990a2 Use ::class to resolve class names in includes files
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I07a925c2a9404b0865e8a8703864ded9d14aa769
2018-01-27 20:34:29 +01:00
Thiemo Mättig
ef470ebf7f Remove @param comments that literally repeat what the code says
These comments do not add anything. I argue they are worse than having
no comments, because I have to read them first to understand they
actually don't explain anything. Removing them makes room for actual
improvements in the future (if needed).

Change-Id: Iee70aad681b3385e9af282d5581c10addbb91ac4
2018-01-10 14:14:26 +01:00
daniel
6af796f3e0 MCR: Deprecate and gut Revision class
This is a re-submission of I4f24e7fbb68.

As a first major step towards Multi-Content-Revisions (MCR),
this patch turns the Revision class into a legacy proxy for
the new RevisionRecord and RevisionStore classes.

Backwards compatibility is maintained for all but some
rare edge cases, like constructing a completely empty
Revision object.

For more information on MCR, see
<https://www.mediawiki.org/wiki/Requests_for_comment/Multi-Content_Revisions>.

NOTE: once this is merged, verify create/delete/restore cycle on beta,
      ideally with emulated replication lag.

Bug: T174025
Change-Id: Ia4c20a91e98df0b9b14b138eb4825c55e5200384
2017-12-21 18:08:54 +00:00
Daniel Kinzler
09bf4f5bb2 Revert "[MCR] Turn Revision into a proxy to new code."
This reverts commit 9dcc56b3c9.

With this patch applied, newly created revisions are sometimes not found
just after submitting an edit, until replicas have caught up.

Our best theory is that it somehow interfere with ChronologyProtector,
but we don't have a good idea how.

Also, as legoktm mentioned, the commit message is terrible and needs fixing.

Change-Id: Idf3404f3fa8f8d08a7fb2ab8268726e2c1edecfe
2017-12-19 12:38:48 +00:00
daniel
9dcc56b3c9 [MCR] Turn Revision into a proxy to new code.
Change-Id: I4f24e7fbb683cb51f3fd8b250732bae9c7541ba2
2017-12-18 14:37:29 +00:00
Addshore
2781a0c1e0 Revert "Remove some remaining mentions of unstubbing from core"
This reverts commit 0f91e75c20.

Bug: T177478
Change-Id: I1937bd116d22479c38a9a34714c2090e2a4c0d7e
2017-10-05 17:27:08 +00:00
Chad Horohoe
0f91e75c20 Remove some remaining mentions of unstubbing from core
Change-Id: Ife34335db7afd9bd4f0936cb90be227d7c4c9651
2017-10-03 23:01:43 +00:00
Umherirrender
3f1a52805e Use short type bool/int in param documentation
Enable the phpcs sniffs for this and used phpcbf

Change-Id: Iaa36687154ddd2bf663b9dd519f5c99409d37925
2017-08-20 13:20:59 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Kunal Mehta
d1cf48a397 build: Update mediawiki/mediawiki-codesniffer to 0.10.1
And auto-fix all errors.

The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.

Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
2017-07-22 18:24:09 -07:00
mdew192837
299255c648 MessageCache: Avoid deprecated wfMemcKey()
Change-Id: I31e3cde0a94e355de9f32a17636aaf54aa2d40c4
2017-05-24 14:15:29 -05:00
Brad Jorsch
6efee6f0db MessageCache: Avoid 'contributions' replicas
On WMF wikis, the partitioning of the revision table on the
'contributions' replicas makes the query here perform really poorly.
Specify 'api' as a hack for now to avoid those replicas.

This query happens once per day per wiki, plus when someone edits the
MediaWiki namespace, so it shouldn't be much additional load.

Bug: T164666
Change-Id: I5ae74d1702144f6475e9cfb13effc43389d66233
2017-05-08 14:00:26 -04:00
Brad Jorsch
f0a80ea778 Prettify MessageCache::loadFromDB()-small query
Adjust the call so it uses JOIN...ON syntax rather than comma joins.

Bug: T164666
Change-Id: I93c8569620d543e9507d0adf1283443915f50d4b
2017-05-08 13:57:13 -04:00
Brad Jorsch
1aac0a2992 Wrap parser output in <div class="mw-parser-output">
This will allow CSS to target just the parser output, without also
accidentally targeting the edit form, diff tables, and so on.

Bug: T37247
Change-Id: If4eb5bf71f94fa366ec4eddb6964e8f4df6b824a
Depends-On: I330c6aa4aaee045614b1801ed34bc9e03be69650
Depends-On: I52a518fa44e017841fe78474012cd69823e0a41d
2017-05-08 05:32:03 +00:00
Aaron Schulz
d735dc562d Move Database and subclasses to Rdbms namespace
Change-Id: I52bef87512f9ddd155d1f4cc0052f6b7a0db5b42
2017-04-12 10:43:57 -07:00
Aaron Schulz
345616cc5e Avoid duplicate key generation code in MessageCache
Also explain the use of !NONEXISTANT in the comments a bit more.

Change-Id: Ib087df9bf31023e7acaf2116e74b2dfeda10c9d2
2017-04-05 23:16:16 -07:00
Brian Wolff
1c7889446d SECURITY: Disable <html> tag on system messages despite $wgRawHtml = true;
System messages may take parameters from untrusted sources. This
may include taking parameters from urls given by unauthenticated
users even if the wiki is a read-only wiki. Allowing <html> tags
in such a context seems like an accident waiting to happen.

Bug: T156184
Change-Id: I661f482986d319cf41da1d3e7b20a0f028a42e90
2017-03-28 21:51:44 +00:00
Aaron Schulz
c2bbb51c20 Make MessageCache use the immutable text cache during cache rebuilds
Only 1 message typically changed per run, so for wikis using
external storage and with many customized messages, this can
make rebuilds considerably faster.

Bug: T158084
Change-Id: Ib668e69a207e3fbeb7871f2f6a102ff1af567368
2017-03-08 20:42:22 -08:00
Aaron Schulz
44ac0de8d1 Set mCache in the second MessageCache::loadFromDatabase() caller
The member variable is needed in the next lines, which previously
just used the array with "LATEST" set and would be seen as invalid
and discarded next time.

Bug: T157033
Change-Id: I5b84b1ae4a9c7b710ee452c61d7d9d6076ec9e6a
2017-02-02 14:06:33 -08:00
jenkins-bot
b550816009 Merge "Inject remaining cache objects into MessageCache" 2017-01-26 03:30:00 +00:00
Aaron Schulz
c962b48056 Avoid races in MessageCache::replace()
Do the process cache update immediately (as before) but push
the shared cache updates to a deferred update. This update
will thus start with a clear transaction snapshot, so it can
acquire the lock before the first SELECT as is proper.

Also added some missing method visibilities.

Bug: T144952
Change-Id: I462554b300d4688b09ab80cd1bb8a4340ffaa786
2017-01-25 00:46:41 +00:00
Aaron Schulz
e99b863eec Inject remaining cache objects into MessageCache
Change-Id: I431d6e9b443a00fdc1b65adb90ae9de496242f81
2017-01-21 06:23:40 +00:00
Jack Phoenix
38c0e0410a Pass the language code to the MessagesPreLoad hook
So that extensions like MessageCommons can try to use this (and
$wgLanguageCode) instead of $wgLang->getCode()/$wgContLang->getCode(), as
the latter ones cause fatals and recursion, at least with the Gadgets
extension also enabled at the same time.

Change-Id: If71fe1ded26c7a1c771128397783783ad5715b00
2017-01-08 02:34:22 +02:00
jenkins-bot
1676448145 Merge "MessageCache invalidation improvements" 2016-12-15 22:18:31 +00:00