Commit graph

239 commits

Author SHA1 Message Date
Aaron Schulz
a556484ba0 Revert "Count API and hook calls, with 1:1000 sampling"
This seems to use 3.67% of index.php real time per xenon.

This reverts commit c4cd239939.

Change-Id: I1eb8dfc72036493e430271f0db3f2f2f96b149c3
2015-08-21 20:22:05 +00:00
Gergő Tisza
c4cd239939 Count API and hook calls, with 1:1000 sampling
Re-adds I6f807adc9cbf71c5d7b83c7eec43965dce1d2a16 and
Ic04daf475b936b942833362c7a979dde671b3ef4 (reverted in
35ccd9c2fe) with 1:1000 sampling
to avoid swamping the statsd hosts.

Also fixes query module logging.

Bug: T102079
Bug: T106450
Change-Id: I8b9366407c0d1713790d08e69aaa518130f01977
2015-08-10 22:57:59 +00:00
Brad Jorsch
9c7385b8ad API: Change default continuation format for action=query
The new format is much easier to use correctly, and should therefore be
the default. The old format is maintained for clients that need the
additional flexibility.

Bug: T96858
Change-Id: I984e63581f4ef23ccada916b6d45131d1fb5e001
2015-06-19 15:13:21 -04:00
Brad Jorsch
e40cbbea65 ApiPageSet: Indicate why a title was invalid
May as well.

Bug: T98198
Change-Id: Ib17088a9685c48d7db647896ecd59aced7911374
2015-05-14 18:34:02 +00:00
Brad Jorsch
1de341dbf9 API: Remove metadata keys from indexpageids output
While we could fix the issue at hand by moving the setArrayType() call,
it's more robust to explicitly remove *all* metadata.

Bug: T96942
Change-Id: I313b19339474ea34880ebef31851eefdf14a7a07
2015-04-23 09:17:56 -04:00
Brad Jorsch
beab6b009e Change API result data structure to be cleaner in new formats
Nothing in this patch should result in changed output for format=json or
format=php except as noted in RELEASE-NOTES-1.25, and changed output for
format=xml should be similar or cosmetic. However, other code accessing
the result data directly may need to be updated.

Bug: T87053
Bug: T12887
Change-Id: I3500708965cb8869b5aed1543381aad208dadd13
2015-04-20 17:49:37 -04:00
Brad Jorsch
1c57794e37 API: Overhaul ApiResult, make format=xml not throw, and add json formatversion
ApiResult was a mess: some methods could only be used with an array
reference instead of manipulating the stored data, methods that had both
array-ref and internal-data versions had names that didn't at all
correspond, some methods that worked on an array reference were
annoyingly non-static, and then the whole mess with setIndexedTagName.

ApiFormatXml is also entirely annoying to deal with, as it liked to
throw exceptions if certain metadata wasn't provided that no other
formatter required. Its legacy also means we have this silly convention
of using empty-string rather than boolean true, annoying restrictions on
keys (leading to things that should be hashes being arrays of key-value
object instead), '*' used as a key all over the place, and so on.

So, changes here:
* ApiResult is no longer an ApiBase or a ContextSource.
* Wherever sensible, ApiResult provides a static method working on an
  arrayref and a non-static method working on internal data.
* Metadata is now always added to ApiResult's internal data structure.
  Formatters are responsible for stripping it if necessary. "raw mode"
  is deprecated.
* New metadata to replace the '*' key, solve the array() => '[]' vs '{}'
  question, and so on.
* New class for formatting warnings and errors using i18n messages, and
  support for multiple errors and a more machine-readable format for
  warnings. For the moment, though, the actual output will not be changing
  yet (see T47843 for future plans).
* New formatversion parameter for format=json and format=php, to select
  between BC mode and the modern output.
* In BC mode, booleans will be converted to empty-string presence style;
  modules currently returning booleans will need to use
  ApiResult::META_BC_BOOLS to preserve their current output.

Actual changes to the API modules' output (e.g. actually returning
booleans for the new formatversion) beyond the use of
ApiResult::setContentValue() are left for a future change.

Bug: T76728
Bug: T57371
Bug: T33629
Change-Id: I7b37295e8862b188d1f3b0cd07f66ac34629678f
2015-04-10 16:57:15 -04:00
Brad Jorsch
6ab6a1ada0 API: Reduce volume of "continue format is changing" warning
It's excessively difficult to determine if the logs of people hitting
this warning are really going to be affected by the change or if they're
making queries where it won't actually matter.

Since 1.26 is coming soon, and with it the decision as to whether to
pull the switch on I984e6358, let's retarget the warnings to those cases
where it's possible it would actually break someone's code.

Change-Id: I91f170fd7d4c791ca8f5d3592c888700f4d5aa97
2015-04-01 08:23:20 -04:00
Brad Jorsch
bfe07bed33 API: Remove explicit profiling
The profileIn/profileOut pair should be covered by the Xhprof profiling
of the method calls it was wrapping.

The profileDBIn/profileDBOut pair are covered by profiling done by the
Database class.

Nothing in extensions in Gerrit is calling anything besides the
profileIn/profileOut pair (and likely those are only to avoid core
formerly throwing exceptions from internal profileDBIn/profileDBOut
calls), and nothing in core or extensions-in-Gerrit is using the methods
for fetching profiling data.

The methods are left as stubs for now to allow for backwards
compatibility in extensions.

Change-Id: I05ba4e2762dc86d5e2bafc183dce701239b43f5c
2015-03-10 18:35:22 -04:00
Max Semenik
b16eb8a42e New hooks ApiMain::moduleManager and ApiQuery::moduleManager
Needed to conditionally register API modules. The previous way, messing with globals
from extension functions, is getting problematic because Config class will make this
impossible.

Change-Id: I86b40aeec555dd6b3cd82cab31d96e85fdf0a665
2015-02-23 10:38:40 -08:00
Aaron Schulz
e369f66d00 Replace wfRunHooks calls with direct Hooks::run calls
* This avoids the overhead of an extra function call

Change-Id: I8ee996f237fd111873ab51965bded3d91e61e4dd
2014-12-10 12:26:59 -08:00
Brad Jorsch
e2055fe0a5 API: Allow generators to return data
It has long been requested that list=search and list=prefixsearch be
able to indicate the search result ordering when used as generators.
This change introduces a generic mechanism to allow for generators to
specify additional page data.

Bug: T16859
Bug: T75623
Change-Id: I115338d2bd890ccc109a79c65f92099c0d41fc2d
2014-11-26 10:41:17 -05:00
Brad Jorsch
991de897e4 API: Split list=deletedrevs into prop=deletedrevisions and list=alldeletedrevisions
list=deletedrevs has always been an odd one: it pretends to be a prop
module sometimes in taking titles from ApiPageSet, but when the pageset
supplies no titles it acts like a list module. This causes problems
such as bug 71389, and makes the whole thing unnecessarily confusing.

The solution is to split the "prop" and "list" aspects into separate
modules: prop=deletedrevisions when input should come from ApiPageSet
and list=alldeletedrevisions when not.

At the same time, let's take advantage of the situation to clear up some
other bugs. And let's share the revision-formatting code with
ApiQueryRevisions instead of partially reimplementing it.

Bug: 23489
Bug: 27193
Bug: 44190
Bug: 71396
Bug: 71389
Change-Id: I3e960d5c655bc57885d6d4ee227e67104808add7
2014-11-04 15:02:03 -05:00
Brad Jorsch
f62bc7536e API: Fix access on getExamplesMessages
ApiBase declares it protected, but for some reason I had made it public
in all subclasses.

Change-Id: I8a50d4f47e66c7f09137968d3941dc5cdc1d28e4
2014-10-29 11:15:27 -04:00
Brad Jorsch
af11268d05 API: Warn when neither rawcontinue nor continue are given to action=query
The default behavior for continuation for action=query will be changing.
Warn clients that they will need to update.

Change-Id: I52d048040e898a0c9630e2aed071b339ec12de42
2014-10-21 19:16:05 +00:00
Brad Jorsch
ad225f501c API: Internationalize all remaining core API modules
This also adds some new ApiBase::PARAM_* constants to generate more
helpful help, and a method to override the default description message
for the use of ApiDisabled and ApiQueryDisabled.

Bug: 71638
Change-Id: Ic0c3d232e0498d58a043037e2e0c6f0b1c3edad3
2014-10-20 16:56:35 -04:00
Brad Jorsch
df457f3809 API: HTMLize and internationalize the help, add Special:ApiHelp
The existing API help, formatted as basically a plain-text document
embedded in XML and with a little bolding and a few links
syntax-highlighted in after the fact, works ok for experienced programmers
but isn't at all newbie-friendly. Further, all the help is hard-coded in
English, which isn't very friendly to non-English speakers.

So let's rewrite it. The help text is now obtained from i18n messages
and output in HTML, with the default display consisting of help for a
single module with links to help for other modules. This, of course,
necessitates deprecating many of the existing help-related methods and
hooks and replacing them with new ones, but backwards compatibility is
maintained for almost everything.

At the same time, action=paraminfo also needs to support the
'description' and other help-related fields being output in wikitext or
HTML, and I11cb063d (to access all modules via the 'modules' parameter
instead of having 'modules', 'formatmodules', 'querymodules', and so on)
is folded in.

And we also add Special:ApiHelp. When directly accessed, it simply
redirects to api.php with appropriate parameters. But it's also
transcludable to allow up-to-date API help text to be included within
the on-wiki documentation.

Note this patch doesn't actually add i18n messages for any API modules
besides ApiMain and ApiHelp. That will come in a followup patch, but for
the moment the backwards-compatibility code handles them nicely.

While we're messing with the documentation, we may as well add the
"internal" flag requested in bug 62905 (although the 'includeinternal'
parameter it also requests doesn't make much sense anymore) and a
"deprecated" flag that's needed by several modules now.

Bug: 30936
Bug: 38126
Bug: 42343
Bug: 45641
Bug: 62905
Bug: 63211
Change-Id: Ib14c00df06d85c2f6364d83b2b10ce34c7f513cc
2014-10-10 10:46:39 -04:00
Brad Jorsch
8e938814c4 API: Add prop modules for backlinks
With the success of prop=redirects, let's create prop=linkshere,
prop=transcludedin, and prop=fileusage.

Let's also add a miser-mode-sensitive 'namespace' parameter to
prop=redirects, since the other three have a 'namespace' parameter using
the new *_from_namespace fields.

Change-Id: Ib650db9988a553771582eed0c183d0cae4b71252
2014-09-18 17:04:42 -04:00
Brad Jorsch
be9a357bee API: Add rawcontinue parameter to action=query
Soon the simplified continuation will become the default for
action=query. This parameter will then serve select the return of the
raw continuation data. To allow for clients to transition without
"unrecognized parameter" warnings, I'd like to add the parameter now
before even adding deprecation warnings.

Change-Id: I5f6895b6a94c53a231a50707ee9463f91295da8f
2014-09-13 14:16:25 -04:00
Brad Jorsch
fdddf94570 API: Overhaul token handling
The current token handling is a mess. This simplifies things greatly:
* *All* tokens are obtained from action=query&meta=tokens, rather than
  being spread over action=tokens, action=query&prop=info,
  action=query&prop=revisions, action=query&prop=recentchanges, and
  action=query&prop=users. All these old methods are deprecated.
* Similarly, there is only one hook to register new token types. All old
  hooks are deprecated.
* All tokens are cacheable.
* Most token types are dropped in favor of a 'csrf' token. They already
  were returning the same token anyway.
* All token-using modules will document the required token type in a
  standard manner in action=help and are documented in machine-readable
  fashion in action=paraminfo.

Note this will require updates to all extensions using tokens.

Change-Id: I2793a3f2dd64a4bebb0b4d065e09af1e9f63fb89
2014-08-26 14:53:45 -04:00
Brad Jorsch
afa153ae9d API: Add a "submodule" param type
This allows action=paraminfo to indicate when a parameter is specifying
a submodule (so clients can detect what the submodule names are). This
will also allow the future action=help HTMLization to easily auto-link
to submodule docs, and will allow ApiSandbox to handle submodules more
generically.

For BC, rather than directly specifying "type": "submodule", it lists
the names of all the submodules as the type and adds an additional
"submodules" indicator on the param info object.

Change-Id: Id31babdc81d970ac781ec11daa3cdafef18ecd5d
2014-08-19 11:25:07 -04:00
Brad Jorsch
f0a6435f3b API: Remove action=paraminfo 'props' and 'errors' result properties
The format for 'props' was never specified and the list for 'errors' is
impossible to keep updated when considering that many errors come from
MediaWiki backend code and extension hook functions. And since there
doesn't seem to be any real use case for either of these, let's just
kill both of them instead of wasting effort on trying to fix them.

Note that neither getResultProperties nor getPossibleErrors are called
from any extensions in gerrit, and none of the other deprecated methods
are called outside of the implementations of those two methods. Removing
the obsolete methods is left to the maintainers of the extensions, as
keeping them hurts nothing and is needed to maintain compatibility with
earlier versions of MediaWiki.

Change-Id: Ie11a401d60c834059fbf1b5625ca8ea093b3337c
2014-08-07 16:51:19 +01:00
Brad Jorsch
d0e16713bc Check for result size failure in ApiQuery
If someone sets $wgAPIMaxResultSize crazily small and then queries a
large number of pages, it can cause ApiQuery to fail to populate the
'pages' node. Then submodules might add data to the node, leading to
errors from the XML formatter (or bogus output from other formatters).

While that particular failure mode is probably going away soon, it's
still good to catch the error situation.

Bug: 68989
Change-Id: If0b2eda421fd86dd7acc4fe48cec47741e214221
2014-08-01 12:26:49 -04:00
umherirrender
02dc9da399 Cleanup some docs (includes/[a-d])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: I7b65fe04db431342cc58b469dc48f41a50c4e891
2014-07-24 19:42:45 +02:00
umherirrender
347a0b294e Fix deprecated of ApiQuery::setGeneratorContinue
No @ before the since

Change-Id: I849b9f950069fe248a6cd86afb1e1beda271f235
2014-07-23 23:36:10 +02:00
Yuri Astrakhan
43b1eb3f72 Added ApiResult::NO_SIZE_CHECK flag for addValue()
This way we no longer need to disable size checking just for one operation
(enable|disable)SizeCheck functions were depricated.

Overall, this is a much better practice than disabling than re-enabling
the flag, as it might lead to accidentally forgetting to re-enable it,
just like the issue with the dangling file handlers, etc.

Example:

disable, do some complex logic, re-enable.  And later, by accident,
the complex logic is changed to return a value half-way, or throws
an exception that gets handled as part of normal operations. This
results in the unsafe disabled state of the result object,
which is not good (tm).

Change-Id: I389a334d35f52f23a1847aca4aef5e96b262f589
2014-07-11 14:53:58 -04:00
Brad Jorsch
1e7ea938da API: Refactor continuation code and add to purge, watch, etc.
While generator support was added to various actions, the continuation
parameters from the generators weren't being returned to the client.

This patch refactors the code for handling continuation out of ApiQuery
and into ApiResult, and then uses it in the other actions that need it.

Bug: 60734
Change-Id: I1a6e51cdc964ddcdd69aaa2f735223371233e8af
2014-07-04 15:48:35 -04:00
Reedy
e48ecbc524 Switch API to use Config classes
Only done where globals are config (so not $wgParser, $wgContLang etc)

Change-Id: Ic39cdd858cfb9096a2bc09618f97e64270d76f13
2014-06-15 23:56:38 +01:00
Thiemo Mättig
23632a4ecd Use precise ApiMain/ApiQuery type hints in all API modules
Which type is used depends on the ApiModuleManager responsible for
the API module. There are two managers, one in ApiMain and one in
ApiQuery. Both contain a list of API modules they instantiate.
Both use $this as the first parameter in the constructors of the
individual modules. There is no other regular way to instantiate the
modules, so we know the type must either be ApiMain or ApiQuery.

The lists don't intersect.

I would have prefered the naming scheme $mainModule for ApiMain
modules and $queryModule for ApiQuery modules but since this
doesn't add much I left the shorter variable names untouched.

Change-Id: Ie6bf19150f1c9b619655a06a8e051412665e54db
2014-05-16 11:07:23 -04:00
umherirrender
e63299d208 Fixed some @params documentation (includes/api)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.
Also added some missing @param.

Change-Id: I758fa4ad80ac95e2ddd3770bcb9b7d2e57ec34ea
2014-04-18 13:55:36 +00:00
Max Semenik
af6d9aba6d New API module, list=prefixsearch
It's pretty much like action=opensearch but can be used as a generator
which can be handy when you need to retrieve both the list of pages and
some information about them.

Change-Id: Iaffe30a0f7402e1316c4885a805692a34bbe1a6a
2014-04-02 19:14:07 +04:00
addshore
719d7a2f03 Add final period to API module descriptions
Change-Id: Icae68c1ab1fd0006e00a3a9a56ae8f831d3d0d45
2014-03-10 09:04:08 +00:00
Brad Jorsch
0ce8a2ac12 API: Add prop=redirects and list=allredirects
While redirects can be sort-of queried using list=backlinks with
blfilterredir=redirects, we can get more accurate results with a module
dedicated to this purpose. We can also get the fragment of the redirect
without having to load the content of the redirect page and parse it.

I'm a bit surprised I was able to put together a query for this that
will work as a prop module. Or did I overlook something?

And then we may as well add the corresponding list=allredirects, to work
like alllinks, allfileusages, and alltransclusions.

Bug: 57057
Change-Id: I81082aa9e4e3a3b2c66cc4f9970a97eed83a6a4f
2014-02-24 11:01:44 -05:00
Brad Jorsch
e06ad68419 API: Add prop=contributors
Certain applications, such as the generation of PDFs, could use a list
of all non-anonymous contributors to the page (as well as a count of
anonymous contributors) without crawling the output of prop=revisions.
This patch adds a prop module to retrieve this information.

Including the IP addresses of anonymous contributors is not realistically
possible without further schema changes, so that is not done here.
Additionally, revisions with DELETED_USER will be skipped entirely.

Change-Id: Iaff50dfb09016154901a5197aa14eb9f8febcbc5
2014-01-07 04:09:47 +00:00
umherirrender
f153998317 Fixed spacing
- Removed double spaces
- Added space after if/switch/foreach
- Removed space on elseif
- Added space around parentheses
- Added newline at end of file
- Removed space before semicolon at end of line

Change-Id: Id40b87e04786c6111e6686d7f7eea1e588bdf37d
2013-11-19 19:03:54 +01:00
Siebrand Mazeland
1707a7995e Fix CodeSniffer errors and warnings (final round for API classes)
Change-Id: I97bf36ebef254818dc17378a06405b681ad19c59
2013-11-15 10:33:50 -08:00
Siebrand Mazeland
c6630e8bbe Update formatting on API classes, Part 3
Change-Id: I526be68e21836201c992b391a81f4d41e1fa801e
2013-11-14 18:12:29 +00:00
umherirrender
cdfd91d85f Add list=allfileusages
This allows the client to enumerate through entries of the imagelinks
table.

This can be useful to find images from foreign repos used on the own
wiki.

Change-Id: I2ad0579e3521868d39f7d495ea9c83a2cd9731d6
2013-10-08 15:32:05 +00:00
Mark Holmquist
d94d29a920 Add meta=filerepoinfo for getting file repo info
We currently only return the image repository name for imageinfo, which
is not particularly useful, especially since image repositories aren't
accessible via the API. This patch adds meta=filerepoinfo, which will
return a list of repositories and relevant information (like the API URL
for ForeignAPIRepo ones)

Also returns the local repo now, and defaults to "all" in a more useful
fashion.

Change-Id: I1aaed0895d2a0bc224c82e93975ecf6afd8cb6b8
2013-09-27 13:53:34 -07:00
umherirrender
5da19f4b3a Add helpurls to some api modules
Change-Id: I41cc20ea5daf089e5fb26d1548d3b58cfe5f4d31
2013-05-16 09:08:18 +02:00
umherirrender
4975761eb2 API: Add new hook 'APIGetPossibleErrors'
This allows extensions to modify the array.

This completes the serie of hooks for similar get methods in ApiBase.

Change-Id: Ib398f5815ab57f25d56356b0997c55a03fd96874
2013-04-19 15:25:58 +02:00
umherirrender
b1bee4e276 Call ApiPageSet::getFinalParamDescription
This allows extensions to modify the param description of the ApiPageSet
params at once for each module, which used the ApiPageSet.

Change-Id: I2ed6f3ad38d3d84182b8525a6b247f721be7f460
2013-04-05 17:09:18 +02:00
Siebrand Mazeland
a72cc6afbc Remove unused local variables
Change-Id: I8eb774c2857dcc87404fd8a7e5fb66c5a4c9643e
2013-04-01 17:42:01 +00:00
Alexandre Emsenhuber
15833dafdf Fix case of some Title methods
Change-Id: I1a37ffb0e8cde4deac027721b0e463585b58d634
2013-03-26 10:54:15 +01:00
Tyler Anthony Romeo
4dcc7961df Fixed @param tags to conform with Doxygen format.
Doxygen expects parameter types to come before the
parameter name in @param tags. Used a quick regex
to switch everything around where possible. This
only fixes cases where a primitve variable (or a
primitive followed by other types) is the variable
type. Other cases will need to be fixed manually.

Change-Id: Ic59fd20856eb0489d70f3469a56ebce0efb3db13
2013-03-11 13:15:01 -04:00
Yuri Astrakhan
0aa24ae558 Fixed many small spelling mistakes and php docs, var decl.
Change-Id: I1508ed7eb77e5e4f700fb63955d626c4f5915840
2013-03-10 23:45:51 -04:00
Yuri Astrakhan
68820277af API continue param to streamline iteration of complex queries
Greatly simplifies query result iteration by the clients
by providing a mechanism to track sub-iterations (props in generated set)

Assuming the client has the param=>value dictionary with the original request
parameters, client will only need to perform this operation in their language
to get all results from the server regardless of what query they make.

  $request = array_merge( $request, $result['continue'] );

Related changes:
* Moved dieContinueUsageIf() from ApiQueryBase to ApiBase
* Internal calls will also return unused param warnings
* Reworked query unit tests for easier testing

Change-Id: Ieb45241fc6db2109f1d92fa3381165ec30701b63
2013-03-01 19:06:46 -05:00
Brad Jorsch
307ac55e50 Add PagesWithProp special page and API module
It would be useful to be able to list pages using a particular page
property, particularly in light of the new Disambiguator extension.

This adds a special page Special:PagesWithProp and an API query module
list=pageswithprop to do just this. It also adds an API query module
list=pagepropnames to list the prop names currently in use on the wiki.

Change-Id: Ib0d4e17f22b8d0cb9894eac6095962315480e809
2013-02-26 00:24:31 +00:00
Yuri Astrakhan
96a9603eea API Remove $wgAPIGeneratorModules - it's dynamic now
* $wgAPIGeneratorModules is now obsolete and will be ignored
* Removed generator tests - obsolete because there is no more list

Change-Id: I014260a42226854a2178345dc3cd0f50b41b3c08
2013-02-19 11:27:13 -05:00
Alex Monk
bf7b7908ea (bug 45017) Check whether this request is a POST before allowing a query module to run, if required by the query module
Change-Id: Ie08fd3a887390e2ead6f6d6fe1e16fa8f8617a2c
2013-02-15 20:54:00 +00:00