Commit graph

62 commits

Author SHA1 Message Date
Umherirrender
7b05197301 Fix caller name in ApiStashEdit::parseAndStash
Seeing {closure} in the logs as caller is not helpful

Change-Id: Id77253bdd4b65b0673214e7bf6ca3cc26bd155fa
2018-09-30 16:01:23 +02:00
Aryeh Gregor
ab94b63526 Improve test coverage for ApiStashEdit
Change-Id: I1b7d95e074a7f101d13398a32898b74145ab4056
2018-08-21 15:32:29 -07:00
Aryeh Gregor
a49a6376be Fix incorrect method name
getTimestamp() returns the timestamp of the revision, and as far as I
can tell that's null here, presumably because we haven't saved the
content and thus there is no revision. getCacheTime() returns the time
when the page was parsed, which is probably what we want.

Change-Id: I7dc446800656236f6ecc872a65e620881e434c54
2018-08-07 16:16:26 +00:00
Aryeh Gregor
6380cadedc Do not return invalid hash from ApiStashEdit
If we were rate-limited, we don't store a new hash, so we shouldn't
return a new one.  If we received a hash from the client, however, that
should still be valid, so we can return it.

Change-Id: Ifc37ae044172b8838fee15d539b076a0d0f02f22
2018-08-06 20:44:20 +00:00
Aryeh Gregor
0438e94222 Clean up param handling in ApiStashEdit
It doesn't make sense to submit both stashedtexthash and text, so
requireOnlyOneParameter() is correct.  This simplifies the code and
gives a more helpful error message. (Previously if both parameters were
passed, we would ignore text unless stashedtexthash was empty, in which
case we would ignore it.)

Change-Id: I306b15eefb6fd4379a3eed88d84113c2e43c4a95
2018-08-06 19:04:43 +00:00
Brad Jorsch
27c61fb1e9 Add actor table and code to start using it
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.

A subsequent patch will remove the old columns.

Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
2018-02-23 10:06:20 -08:00
Aaron Schulz
6fb6fc9dc2 Avoid blocking locks during API edit stash generation
Bug: T180793
Change-Id: Ibc29a20329113c3ee54fd48cfe86ca8251e8098d
2017-11-21 19:23:41 +00:00
Chad Horohoe
b0944a1371 Fix up logging grouping in ApiStashEdit
Change-Id: Ie3d7b465faa26966c5745ca1bb09843a26193b9e
2017-11-09 20:19:30 +00:00
Aaron Schulz
23cd890fc9 Sanity check "stashedtexthash" param before checking memcached
The makeKey() method should do escaping and shortening, but it
is safest and clearer to not rely on that.

Change-Id: Ia4a95e0bb12074d141ddcca2089b920403cab100
2017-08-31 14:35:17 -07:00
Aaron Schulz
7ff8529984 Avoid high edit stash TTLs when a user signature was used
This adds a new ParserOuput user-signature tracking flag.

Bug: T84843
Change-Id: I77de05849c15e17ee2b9b31b34172f4b6a49a38e
2017-07-06 16:34:26 -07:00
Aaron Schulz
82e2c924e4 Remove "@author Aaron Schulz" annotations
Bug: T139301
Change-Id: Ib5248e8e27d60611c7373bce4b29dd5e85aa3489
2017-06-27 15:24:14 -07:00
Erik Bernhardson
d67197fa11 Cleanup some incorrect return annotations
Most of these are simply changing annotations to reflect
reality. If a function can return false to indicate failure
the @return should indicate it.

Some are fixing preg_match calls, preg match returns 1, 0 or false,
but the functions all claim to return booleans.

This is far from all the incorrect return types in mediawiki, there
are around 250 detected by phan, but have to start somewhere.

Change-Id: I1bbdfee6190747bde460f8a7084212ccafe169ef
2016-12-12 10:15:05 -08:00
Brad Jorsch
4e6810e4a2 API: i18n for warnings and errors
API warnings and error messages are currently hard-coded English
strings. This patch changes that.

With a few exceptions, this patch should be compatible with non-updated
extensions:
* The change to ApiBase::$messageMap will blow up anything trying to
  mess with it.
* The changes to the 'ApiCheckCanExecute' hook will cause a wrong
  (probably unparsed) error message to be emitted for extensions not
  already using an ApiMessage. Unless they're currently broken like
  Wikibase.

Bug: T37074
Bug: T47843
Depends-On: Ia2b66b57cd4eaddc30b3ffdd7b97d6ca3e02d898
Depends-On: I2e1bb975bb0045476c03ebe6cdec00259bae22ec
Depends-On: I53987bf87c48f6c00deec17a8e957d24fcc3eaa6
Depends-On: Ibf93a459eb62d30f7c70d20e91ec9faeb80d10ed
Depends-On: I3cf889811f44a15935e454dd42f081164d4a098c
Depends-On: Ieae527de86735ddcba34724730e8730fb277b99b
Depends-On: I535344c29d51521147c2a26c341dae38cec3e931
Change-Id: Iae0e2ce3bd42dd4776a9779664086119ac188412
2016-12-06 10:20:48 -05:00
Kunal Mehta
61adc1e146 Use namespaced ScopedCallback
The un-namespaced \ScopedCallback is deprecated.

Change-Id: Ie014d5a775ead66335a24acac9d339915884d1a4
2016-10-17 15:46:05 -07:00
Aaron Schulz
ae648472b0 Fix ApiStashEdit IDEA warnings
Change-Id: Icc1d0225908e432df10c36d69c6ed282f8761198
2016-09-07 12:00:52 -07:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
3853783ef7 Optimize summary-based extension edit stash caches
* Send stash requests when the summary changes, so that things like
  AbuseFilter caching have a higher hit rate.
* Make the backend API skip parsing if a fresh cache is already present.
  This makes requests for summary-only changes much faster and more likely
  to finish in time.
* Avoid sending the full text if only the summary changed since the
  last successful stash. This works via an optional stashedtexthash
  parameter to the API.
* Also always apply the lock in parseAndStash(), even for VE.

Change-Id: I9bfd74cf05411853b675c6f54ff5d8934bcfc54c
2016-08-10 17:50:19 -07:00
Aaron Schulz
005b4d6fff Try to predict the rev_id when preparing edits
During both the edit stash and first parse in on page save,
guess what the rev_id will be and use that instead of null.
Only reparse if it turns out to be wrong. This avoids extra
parsing on wikis that have low-medium traffic, and does not
cost much. The parsing that can be avoided is:
a) in doEditContent() by using the stash
b) in doEditUpdates() by using the doEditContent() result,
   whether that was able to use the stash or not itself

Also improved the parse operation logging in save paths.

Bug: T137900
Change-Id: Ic6faae70a78b4e223e4d3585cefd482c0fa00677
2016-06-29 05:39:33 -07:00
Aaron Schulz
6b192e6146 Improve edit stashing when vary-revision is used
At least avoid the first parse in doEditContent()/filters which
never has the revision set either. The second parse cannot be
avoided in doEditUpdates() however.

Bug: T136678
Change-Id: I12d2c3bbe1b21ad2ed9a484745d976ad62475e0d
2016-06-17 16:21:45 -07:00
Aaron Schulz
406d6a2b46 Perform edit stashing when the edit preview or diff form is requested
* Users are likely to save after they inspect and review their changes.
  The buttons to do so are also located below the edit summary box.
* This obsoletes the backend stashing on preview. Most of those parses
  would have been useless due to being per-section only. Also, some
  extensions like the Graph extension disable stashing for "preview"
  output anyway. Simplify the code by removing that method.

Bug: T136678
Change-Id: Ied77bdbd191dd9267d4295b0fa7b942f65b062db
2016-06-14 18:25:35 +00:00
Aaron Schulz
cb68ff0230 Send the edit summary/user to the edit stash API
This can be useful for pre-caching AbuseFilter check results

Bug: T137698
Change-Id: I03e3924408a2a5da09a411b66d09958a1b78cb99
2016-06-13 22:40:50 +00:00
Aaron Schulz
a74346b9bb Add more logging info to parseAndStash() for uncacheable content
Change-Id: I2409f0db518d366a747aa01ba5593dccdb34e68c
2016-06-09 21:03:59 -07:00
Aaron Schulz
e40b678de0 Simplify ApiStashEdit::checkCache()
Per https://grafana.wikimedia.org/dashboard/db/edit-stash, the
proven_* cases are almost never hit. Simplify the code to avoid
complexity and slow link table queries.

Change-Id: I3f8bd79a80ddb26c6188b1a301c08766e38f6ad4
2016-06-09 21:02:40 +00:00
Aaron Schulz
6f86660e23 Include title/user in some edit stash log messages
Change-Id: I49d61d9cfc5edacf415f6822cb50cc9007735a27
2016-06-09 07:16:17 +00:00
Aaron Schulz
27e6e29888 Remove getOptions() checks from getStashKey() entirely
It only avoided a tiny edge case of options lag, which is
not worth the complexity.

Change-Id: I94bc35fc89eb46436a23b10aee1b7d339859c7ea
2016-06-06 17:27:37 -07:00
Aaron Schulz
58886b169a Make edit stash keys less preference sensitive
Only factor in the user ID/name/sig instead of anything that
might happen to touch the user row.

Bug: T136678
Change-Id: I004790a017c68cfcff6729f6fbfd0f79b6904c20
2016-06-06 22:03:11 +00:00
Aaron Schulz
5051edd835 Send edit stash metrics for cache attempts
Change-Id: I22d5626cca55c2cc35e1b33fced5902e8d364d40
2016-05-26 21:02:39 +00:00
Aaron Schulz
d690083213 Bail out in ApiStashEdit for bots for sanity
checkCache() is disabled in this case anyway.

Change-Id: I1c18585eecc1a7c4c0e24546799c7ee448b3ea57
2016-05-26 19:59:16 +00:00
Aaron Schulz
cddfe0d98f Exclude bots from edit stash stats
Change-Id: Id309879eaaf039de7ac5f531930fa8e758a959c7
2016-05-26 11:36:40 -07:00
Aaron Schulz
ba6844e21c Improve edit stash hit rate for logged-out users
Check whether they made intervening edits just like logged-in users.

Bug: T134620
Change-Id: Id1e0808caee0e474570c4f2e4b1cf845572e17e0
2016-05-17 19:04:42 +00:00
Aaron Schulz
2be38b3670 Make "presumed-fresh" edit stash case cover when users make intervening edits
This still handles users editing inclusions themselves, but is
more relaxed in terms of edits by other users to the inclusions.

Bug: T134620
Change-Id: I6c0d189957481dfb6da0e73581b1d5b69fd3a352
2016-05-13 21:20:41 +00:00
Aaron Schulz
07e23b39a4 Make stashEditFromPreview() call setCacheTime()
This makes sure the output can pass the "presumed fresh" check.

Bug: T134620
Change-Id: I4f9ac2f1b9cef5c6b179946ea02a5e2fad2adc09
2016-05-07 11:02:35 +00:00
Aaron Schulz
6c3780e1a1 Bump PRESUME_FRESH_TTL_SEC to improve hit rate and avoid link queries
Task: T134620
Change-Id: I3f76d81ee0ff42bbab6a1b000907e80f4533519c
2016-05-07 00:39:45 +00:00
Aaron Schulz
af349f79c8 Fix timestamp check in ApiStashEdit::checkCache
* The wrong time method was used, and it also was not set.
* Bumped the threshold a bit while at it, which was basically
  0-5 minutes before.

Bug: T133332
Change-Id: Ide3e66f551aa6e50410c562e5c917141d59b7f64
2016-04-28 22:01:03 +00:00
Ori Livneh
0f1ffa4da7 Segment stash edit cache stats by basis for hit/miss
Instead of just counting cache hits and misses, segment the counts by reason,
so we can differentiate (for example) timestamp-based cache hits from
staleness-check-survivor cache hits. I want this data so I can determine
whether increasing the cutoff for timestamp-based hits from 3 to 5 seconds has
a substantial enough impact to warrant the slightly weaker consistency.

Also changed 'cache-hit' to 'cache_hit'. MediaWiki normalizes the dash to an
underscore anyway, but the normalization is there for dynamically-constructed
key names (or name segments). In the case of hard-coded values, it is desirable
for the code to be as close as possible to the final form of the metric name,
to simplify metric lookup.

Change-Id: I0cd61da9746e3ca3695e23200f698b8b1371798c
2016-04-19 01:31:39 +00:00
jenkins-bot
2aff530dfb Merge "Use single quotes in API where possible" 2016-03-08 17:03:24 +00:00
Siebrand Mazeland
bc0ae710e3 Use single quotes in API where possible
Change-Id: I972e296f4820f78f5dfcecc27bc4912ca84a3178
2016-03-08 17:27:00 +01:00
Siebrand Mazeland
e5b5fe7ba3 Declare functions with access modifiers
Parents have this visibility.

Change-Id: Icb4d0a112def0ac02f9aa42723c81bc63c085450
2016-03-08 16:16:33 +00:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Aaron Schulz
d478d4a154 Add ParserOutputStashForEdit hook for extension cache warming
This can pre-cache slow queries by extensions that happen on
edit submission.

Bug: T116557
Change-Id: I803f69013f68e80a53dd3c466bddff3ebe2b659b
2016-01-26 17:23:56 -08:00
jenkins-bot
c102eb8cc0 Merge "Remove unused ApiStashEdit variable" 2015-12-18 20:08:39 +00:00
Aaron Schulz
f4a07ea296 Make ApiStashEdit use statsd metrics
Change-Id: Iffdeae7f867490e3c69c4b6d79a5f844cb7fce23
2015-12-18 02:59:55 +00:00
Aaron Schulz
83c7d3fbb9 Remove unused ApiStashEdit variable
Change-Id: Ifd85dd6b96e80d7c1163566153e8d746b27754d8
2015-12-15 21:23:23 -08:00
Aaron Schulz
9c919dd2cf Make edit stashing use named DB locks
These should be faster by avoiding the inefficient back-off
polling in BagOStuff::lock().

Change-Id: I013b2567716eea5e354e13f3e9640573886e596a
2015-12-15 20:32:18 +00:00
Bryan Davis
b89a6a5944 Convert ApiStashEdit to PSR-3
Convert the wfDebugLog calls in ApiStashEdit to direct use of the PSR-3
logging layer and add severity information for each message.

Change-Id: Ic91e89ceee405a1d22e5e3c461ad37795cae4a6f
2015-12-04 15:18:07 -07:00
Aaron Schulz
92bbfd5ea6 Migrate more callers away from $wgMemc
Callers should use more expliciy lazy-loaded
cache factory methods.

Change-Id: Ifa0bf389720a09a931ee6466b993f787d83a09a7
2015-10-26 19:07:12 +00:00
Brad Jorsch
ec658adad5 Add i18n messages for ApiStashEdit
Bug: T113189
Change-Id: Iee7b2fbb7b9cd4e7255c87d78be9396073f41236
2015-09-21 10:31:01 -04:00
Aaron Schulz
84c4e7597e Ignore some ScopedCallback IDE warnings
Change-Id: Iefe24ee5dae90d70c2acaaae9dc5d44c1758d53a
2015-07-15 14:13:42 -07:00
Aaron Schulz
6ed8416a70 Fixed edit stash inclusion queries
* These are hit when the stashed edit is several seconds
  old. The old code was not using makeList() correctly.

Bug: T105597
Change-Id: I265307715996c50d819915a55ea34bbc0ed6c5c9
2015-07-13 21:46:01 +00:00
Aaron Schulz
5399fba68b Use less fuzzy User::getDBTouched() in ApiStashEdit::getStashKey()
* The getTouched() value can bump more often or vary in DCs

Change-Id: Idffa22c9dde7c7950943b104ed61f5779636b7e7
2015-06-02 20:31:14 +00:00