Commit graph

520 commits

Author SHA1 Message Date
Derick Alangi
f9f2809a38 SearchEngine: Hard deprecate SearchEngine::transformSearchTerm()
As a follow up of it's soft deprecation in 1.32, this method should
be hard deprecated and removed. Also, remove related code that calls
this method.

Usage:

https://codesearch.wmflabs.org/search/?q=(%3A%3A%7C-%3E)transformSearchTerm&i=nope&files=&repos=

Bug: T221431
Change-Id: I623b39e7ec791a66c4cf40178edb80b2ebf4bb06
2019-04-26 19:24:02 +00:00
jenkins-bot
37344b9daf Merge "Define index types as strings" 2019-04-23 08:57:59 +00:00
Stanislav Malyshev
9e035a229e Define index types as strings
This will allow to use them easier in JSON config files.

Bug: T142809
Change-Id: I4053a87fe8e53c9b506604db6fad2f98f7dba150
2019-04-18 23:59:18 -07:00
Derick Alangi
0a4c2670a1 search: Remove unused deprecated methods replaced by MW Services
The following methods deprecated in .27 and no longer used have
been removed;

- SearchEngine::userNamespaces()
  https://codesearch.wmflabs.org/search/?q=%3A%3AuserNamespaces&i=nope&files=&repos=

- SearchEngine::namespacesAsText()
  https://codesearch.wmflabs.org/search/?q=%3A%3AnamespacesAsText&i=nope&files=&repos=

- SearchEngine::create()
  https://codesearch.wmflabs.org/search/?q=SearchEngine%3A%3Acreate&i=nope&files=&repos=

- SearchEngine::getSearchTypes()
  https://codesearch.wmflabs.org/search/?q=%3A%3AgetSearchTypes&i=nope&files=&repos=

- SearchEngine::getNearMatch()
  https://codesearch.wmflabs.org/search/?q=SearchEngine%3A%3AgetNearMatch%5C(&i=nope&files=&repos=

Also, replace deprecated use of searchableNamespaces() in SearchEngine.

Bug: T220656
Change-Id: I330fbfea158f70b2328da7c850fdb833e39b6c7d
2019-04-19 00:09:42 +00:00
James D. Forrester
f84457f8bd SearchEngine: Hard deprecate unused static methods deprecated in 1.27
Change-Id: I96c4454a2f89df0be5de20ca0506c3ce2c75cf7c
2019-04-18 16:54:30 -07:00
Fomafix
8a314d9b43 Simplify by using ?: operator
Change-Id: I2851cc51c9e05dd0599733be5af39e19f12b52e2
2019-04-15 15:05:00 +02:00
jenkins-bot
7f2f49ad23 Merge "Improve documentation of constants throughout the codebase" 2019-04-12 21:38:16 +00:00
Aaron Schulz
449ad97224 search: make legalSearchChars() non-static
No outside callers from code in gerrit are using this statically

Change-Id: I9513a5384bca71f419a9af09908678bcc86c0263
2019-04-11 17:55:54 -07:00
jenkins-bot
b306873222 Merge "Remove references to field rev_text_id" 2019-04-09 15:25:05 +00:00
Kunal Mehta
4ef179e335 Fix/suppress misc phan errors (#5)
Add lots of missing return statements, or remove incorrect doc blocks.

Change-Id: I0881e98fbb9d0d4cf79ecc824064d24538055d3f
2019-04-05 15:53:37 -07:00
Umherirrender
6745cb572a Move class SearchEngineDummy to own file
Each class should have its own file, even it is a no-op

Change-Id: I90124ed35e354c546fcd69d52dc1febb71b2c71f
2019-03-29 20:35:35 +01:00
Umherirrender
8d5a8fb181 Move PrefixSearch classes to own files
Move all into search subfolder

Change-Id: I4097b9745d22545afe6761da1fa67cf98af5b811
2019-03-27 21:40:33 +01:00
Bill Pirkle
8f03082521 Remove references to field rev_text_id
Field rev_text_id is being retired as part of MCR Schema Migration.
Remove references to this field from the Postgres search queries.

Bug: T198341
Change-Id: Id3cbf853aacc06709f441a93248b55943097cd11
2019-03-26 10:41:29 -05:00
Thiemo Kreuz
18741b9306 Improve documentation of constants throughout the codebase
The most notable improvements I was able to fit into this patch can be
seen in the User class, as well as in AbstractRestriction.

Our documentation generator ignores the @const tag. It's not needed. Just
have a comment above a constant and it will show up in the generated
documentation.

Using @var is misleading because a constant is not a "variable". The type
of a constant is strictly derived from it's value. Documenting the type
typically does not provide useful information. Doxygen does not understand
the type, but ignores any @… tag and renders everything else as plain text.

I can split this patch if you prefer. Please tell me.

Change-Id: I8019ae45c049822cdc1768d895ea3e3216c6db5f
2019-02-26 14:02:50 +01:00
James D. Forrester
5cc3a2ae4e Drop SearchEngine::getNearMatchResultSet(), deprecated in 1.27 and unused
Change-Id: I87987af702b9b92c4d1b3578e69f1b2e68529b52
2019-02-09 08:43:30 +00:00
Thiemo Kreuz
734a969d55 Safe replacement of a lot of !count() with === []
This was originally a global search and replace. I manually checked all
replacements and reverted them if (due to the lack of type hints) either
null (that would be 0 when counted) or a Countable object can end in the
variable or property in question.

Now this patch only touches places where I'm sure nothing can break.

For the sanity of the honorable reviewers this patch is exclusively touching
negated counts. You should not find a single `!== []` in this patch, that
would be a mistake.

Change-Id: I5eafd4d8fccdb53a668be8e6f25a566f9c3a0a95
2019-01-15 17:28:49 +01:00
Max Semenik
1da14005d9 Hard deprecate SearchEngine::getNearMatchResultSet()
Deprecated in 1.27, no callers.

Change-Id: I6084a114fe8d0b4681e2e4b55eada06b684c719c
2018-12-08 22:23:24 -08:00
jenkins-bot
bd78869618 Merge "No yoda conditions" 2018-12-09 01:34:23 +00:00
Jakub Vrana
45435d05a5 Change typehint callback to callable
Found by PHPStan.

Change-Id: I09f21da69ad9b9357cee85a47717dbe1bca04070
2018-12-01 10:10:41 +01:00
Fomafix
3ee1560232 No yoda conditions
Replace
  if ( 42 === $foo )
by
  if ( $foo === 42 )

Change-Id: Ice320ef1ae64a59ed035c20134326b35d454f943
2018-11-21 17:54:39 +01:00
jenkins-bot
5466734477 Merge "doc: Modernise parameter names and documentation for 'replica' DBs" 2018-11-13 21:39:14 +00:00
jenkins-bot
fbc9e214e7 Merge "Add docstrings for text search index field types" 2018-11-01 10:21:55 +00:00
Adam Wight
861d718768 Add docstrings for text search index field types
Change-Id: Icd34ec79db4d8fd024a82045aaa9bedb6556caa8
2018-10-31 17:30:58 -07:00
James D. Forrester
903e8b63de doc: Modernise parameter names and documentation for 'replica' DBs
Non-breaking change. Remaining uses are public interfaces (a constant, two
globals, a config sub-parameter, SQL queries, storage function names), one i18n
message key, and a whole lot of maintenance scripts with calls to the deprecated
function wfWaitForSlaves().

Change-Id: I6ee5ca92ccf6a80c08f53d9efe38ebb4b05064d7
2018-10-31 10:36:48 -07:00
Fomafix
43244db9a2 Use PHP 7 '??' operator instead of if-then-else
Change-Id: If9d4be5d88c8927f63cbb84dfc8181baf62ea3eb
2018-10-21 21:46:46 +02:00
Fomafix
5632815976 Write Latin and other scripts with captial letter
Change-Id: I16c660e54191b63cd6eb3407cb00504665930c4e
2018-10-05 18:49:08 +02:00
Erik Bernhardson
b357416273 Go search to consider fragment only title invalid
When submiting using the 'go' feature of Special:Search (standard
usecase for autocomplete) a nearby title can be selected.
Unfortunately a fragment only title is considered valid, which
sends the user to Main_Page#searchterm.

Instead don't allow a search starting with # to trigger the go
feature.

Bug: T182452
Change-Id: I8ce3ce8633e23db41ae6eafb9996ca7294f8445a
2018-09-11 13:03:53 -07:00
Zoranzoki21
1f2e9c1b54 Fix common typos in code
Bug: T201491
Change-Id: Id962b79f2590c51380cb977e727b7548abc11d33
2018-09-02 09:03:07 +00:00
jenkins-bot
9171317645 Merge "Add Special:Search sort parameter without ui" 2018-08-21 14:57:13 +00:00
Umherirrender
40d3ce8d17 Reduce calls to MediaWikiServices::getInstance()
In some functions MediaWikiServices::getInstance() was called twices or
in loops. Extract the variable to reduce calls.

Change-Id: I2705db11d7a9ea73efb9b5a5c40747ab0b3ea36f
2018-08-18 06:02:39 +02:00
Erik Bernhardson
c6a83871b5 Add Special:Search sort parameter without ui
Search engines support the sort parameter, it would be nice if we
could pass it to be rendered in the web ui. This avoids implementing
any ui, which can be done at a later time.

Bug: T195071
Change-Id: Id7bd35bd4259bb1f1f856933d279dbd6eddfa2e2
2018-08-16 17:12:05 -07:00
Aryeh Gregor
90d4f56fe4 Mass conversion of $wgContLang to service
Brought to you by vim macros.

Bug: T200246
Change-Id: I79e919f4553e3bd3eb714073fed7a43051b4fb2a
2018-08-11 22:44:29 -06:00
Timo Tijhof
64892ce978 search: Use self::class instead of __CLASS__ in SearchResultSet
For ease of understanding, use the same technique for both
sides of the comparison. Follows-up c2a308075f.

Change-Id: I66475fd4baaa6ab7cd24ad884e9fe01a1cd30d9f
2018-08-08 01:08:37 +01:00
David Causse
cf41c48f8d Deprecate SearchEngine::replacePrefixes
This should be handled internally by SearchEngine implementations.

Bug: T198860
Change-Id: Ifbfd0fcb81fcacf5228bd2ffcac7b80fca872b2a
Depends-On: I7d4ff9498fa1f4ea66835c634b8931f4c29993fb
2018-07-17 21:56:14 +00:00
David Causse
3115b3202d Unify SearchEngine normalizeNamespace and parseNamespacePrefixes
These methods are very similar there should be no need to have
two differents way to extract the namespace prefix.

Bug: T198860
Change-Id: I22802278452559d35a3d8f6068549c1fef1a5e86
2018-07-17 21:56:05 +00:00
David Causse
46c17ddb1f Deprecate usage of SearchEngine:transformSearchTerm
This method was introduced in 4115586000
to support the prefix URI param introduced by the InputBox extension.
There are no reasons that this logic is exposed to SearchEngine users
and should be handled internally by SearchEngine implementations
that supports it.
Previously the search query was updated, now the prefix param will passed
along using SpecialSearch::$extraParams.

Bug: T198318
Change-Id: I33518d3f3ddee741ff4f3b47eb4928009bea66d1
Depends-On: I67c7f1886dd6a2d07c12015e2711c138e9f140e9
2018-07-17 14:41:08 -07:00
jenkins-bot
ef97002179 Merge "Simplify PHP by using ?? and ?:" 2018-07-11 09:09:18 +00:00
Fomafix
6866cfec37 Simplify PHP by using ?? and ?:
Also remove not necessary surrounding parentheses.

Change-Id: I0eb5c9c1bdfb09a800258379cdcefb5fd4d3d21c
2018-07-10 20:03:17 +00:00
C. Scott Ananian
0935e47a72 Remove most uses of deprecated Language::truncate()
The Language::truncate() function was split into
Language::truncateForVisual() (which measures characters) and
Language::truncateForDatabase() (which measures bytes) in 1.31, but
the patch which soft-deprecated Language::truncate() didn't actually
remove all the uses in the codebase.  Replace most of those old uses
now, which should actually improve the situation for
non-latin-alphabet users who were getting unfairly squeezed in a
number of places.

Bug: T197492
Change-Id: I2291c69d9df17c1a9e4ab1b7d4cbc73bc51d3ebb
2018-07-09 13:36:27 -04:00
Umherirrender
130ec2523d Fix PhanTypeMismatchDeclaredParam
Auto fix MediaWiki.Commenting.FunctionComment.DefaultNullTypeParam sniff

Change-Id: I865323fd0295aabd06f3e3c75e0e5043fb31069e
2018-07-07 00:34:30 +00:00
Umherirrender
ae2f5d6232 Add missing use statements
Swap some ResultWrapper/Database to interface IResultWrapper/IDatabase

Change-Id: I312cb3cca5742f72efdb0965ba35362f2eb1129d
2018-06-27 20:49:23 +02:00
Erik Bernhardson
83bae78c3a Silently drop unknown titles in completion search
This mimics how full text works by silenty dropping results returned from
search that no longer exist. This could be because the search index is slightly
out of sync with reality, or the search engine could simply be broken.

Only silent from the users perspective. We maintain a count in statsd of
the number of titles dropped. This can be monitored over time to
recognize any increases.

Bug: T115756
Change-Id: I2f29d73e258cd448a14d35a2b4902a4fb6f61c68
2018-06-11 13:56:19 -07:00
Erik Bernhardson
2a43939ffb Push pagination decision for search into SearchEngine
Various code using the search engine shouldn't need to implement it's
own methods, such as over-fetching, to determine if there are more
results available. This should be knowledge internal to search that is
exposed by a boolean.

Change-Id: Ica094428700637dfdedb723b03f6aeadfe12b9f4
2018-06-11 13:35:44 -07:00
Erik Bernhardson
c2a308075f Convert SearchResultSet to typical iteration
The funky iteration here was at best annoying. Switch
it over to an iterator based approach with appropriate
BC code to simulate the old iteration style.

Depends-On: I19a8d6621a130811871dec9335038797627d9448
Change-Id: I9fccda15dd58a0dc35771d3b5cd7a6e8b02514a0
2018-06-11 13:35:41 -07:00
jenkins-bot
cc0fe6c4a7 Merge "Deprecate overriding SearchEngine::search*" 2018-05-16 13:31:56 +00:00
jenkins-bot
bbe9917203 Merge "Make internal search methods private for db implementations" 2018-05-15 20:09:59 +00:00
Erik Bernhardson
fdc133ef1a Deprecate overriding SearchEngine::search*
The plan is to convert these methods into final, considering
it a removal under the deprecation policy. By making entry
points into the search engine final we provide a guaranteed
point where generic handling can be applied to all search engines.

The first use case for this generic handling is pushing pagination
via overfetch into the SearchEngine class instead of re-implementing
an overfetch in individual parts of the code that perform searches.

Change-Id: I3426d6a2f32d8b368b044b154e1cb70dac007c62
2018-05-15 08:28:28 -07:00
David Causse
39cbbab021 Allow 'all:' on all wikis in addition to 'searchall' translation
This allows to have a common syntax useable everywhere.

Bug: T165110
Change-Id: If71fe5df045fb754925946088f8f793197bc8301
2018-05-11 15:20:27 +02:00
David Causse
0824579606 Make internal search methods private for db implementations
Change-Id: I622f1b35e27de7918e32c4ddd74ceb4e257c5b02
2018-05-09 11:56:27 +02:00
runntb
b033d3dc34 search: Add result ranking in MySQL
MySQL normally attempts to rank the results when performing a full-
text search. However, this behavior is disabled when using BOOLEAN
MODE. While BOOLEAN MODE is needed in the WHERE clause, the default
NATURAL LANGUAGE MODE can be used in an ORDER BY clause to reenable
ranking.

Bug: T192458
Change-Id: I09462c339432927efead58fb543a10aed2c53195
2018-05-01 17:00:53 -04:00
James D. Forrester
5f018dd6a4 Drop OpenSearch::getOpenSearchTemplate(), deprecated in 1.25
Change-Id: Ib76b96cf392b7f9fa38d28173dd2cd170e08a881
2018-03-08 10:43:28 +00:00
Reedy
39f0f919c5 Update suppressWarning()/restoreWarning() calls
Bug: T182273
Change-Id: I9e1b628fe5949ca54258424c2e45b2fb6d491d0f
2018-02-10 08:50:12 +00:00
Umherirrender
3124a990a2 Use ::class to resolve class names in includes files
This helps to find renamed or misspelled classes earlier.
Phan will check the class names

Change-Id: I07a925c2a9404b0865e8a8703864ded9d14aa769
2018-01-27 20:34:29 +01:00
Stanislav Malyshev
030da07b18 Use getSize since SearchSuggestionSet does not implement Countable
Bug: T184934
Change-Id: I39459352399e2023149b715b049084826df22935
2018-01-22 12:13:34 -08:00
jenkins-bot
a18476eab3 Merge "Remove @param comments that literally repeat what the code says" 2018-01-11 23:48:03 +00:00
Thiemo Mättig
ef470ebf7f Remove @param comments that literally repeat what the code says
These comments do not add anything. I argue they are worse than having
no comments, because I have to read them first to understand they
actually don't explain anything. Removing them makes room for actual
improvements in the future (if needed).

Change-Id: Iee70aad681b3385e9af282d5581c10addbb91ac4
2018-01-10 14:14:26 +01:00
Thiemo Mättig
fa15c98787 Improve (weak and strict) type hints invarious places
Change-Id: I3a42ec1547fae971d7b495c99dd144739d8521ab
2017-12-28 16:31:56 +01:00
Huji Lee
e74bfe13f6 Require indentation of CASE statements in PHP code
Bug: T182546
Change-Id: I91a9555893a08e4ec58da97c6cc4d1e70000ff6b
2017-12-10 22:07:50 -05:00
jenkins-bot
03cd9495a4 Merge "Fix a few minor mistakes in PHPDoc tags" 2017-09-15 19:18:17 +00:00
Thiemo Mättig
7301b2514e Fix a few minor mistakes in PHPDoc tags
Change-Id: I2d5f876c9945ba21f8748562230170e3c7ea2ff2
2017-09-15 11:17:17 +02:00
Umherirrender
f739a8f368 Improve some parameter docs
Add missing @return and @param to function docs and fixed some @param

Change-Id: I810727961057cfdcc274428b239af5975c57468d
2017-09-10 20:32:31 +02:00
Umherirrender
9b8b314992 Fix spacing for @param and indent of function comments
In phpcs.xml rename renamed sniffs and add the failing sniffs,
because now the whole sniff is no longer excluded.

Change-Id: If5b0bd16028761abc2c47ace9e97d37ad14bb36f
2017-08-15 14:33:29 +00:00
jenkins-bot
cc4115e87f Merge "Add missing @param and @return documentation" 2017-08-11 21:32:58 +00:00
jenkins-bot
84b6d5c2e5 Merge "Add missing type to @param documentation" 2017-08-11 21:31:51 +00:00
WMDE-Fisch
6df9ed1ad6 update mediawiki-codesniffer to 0.11.0 and fix issues
- mostly auto fixes
- some too long lines fixed
- ignore amp space in one case  passing by reference

Change-Id: I6472f83bc3cbf4bd629d83050cc3319b19ec465c
2017-08-11 22:27:51 +02:00
Umherirrender
718e63694d Add missing @param and @return documentation
Change-Id: I1d1098eec3933df6561cceef646576013ddc08c8
2017-08-11 22:17:01 +02:00
Umherirrender
5544cef16b Add missing type to @param documentation
Change-Id: I6b2c9c7af9a281fe457099cc3a336a60a25e74aa
2017-08-11 20:37:35 +02:00
Umherirrender
ace44e2064 Use correct variable name in @param documentation
For some varargs a variable name is added with suffix ,... as seen for
many other varargs

Some @param are swapped, because there are in the wrong order

Enable Sniff MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

Change-Id: I60fec6025bce824d5c67563ab7b65ad6cd628ad8
2017-08-11 19:27:19 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Umherirrender
bf61a77431 Change @inheritdoc to @inheritDoc
Only @inheritDoc works for the Sniff
MediaWiki.Commenting.FunctionComment

Change-Id: I91fc02cda6701d790e4334fc2bc47f230955545c
2017-08-11 16:49:52 +02:00
jenkins-bot
e72303c9f3 Merge "Remove auto-generated "Constructor" documentation on constructors" 2017-07-21 13:19:44 +00:00
Thiemo Mättig
91a920fd85 Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

This is similar to I994d11e. Even more trivial, because this here is
about comments that don't say anything but "constructor".

Change-Id: I474dcdb5997bea3aafd11c0760ee072dfaff124c
2017-07-21 12:19:30 +02:00
jenkins-bot
598a250b71 Merge "Support custom offsets from SearchResultSet" 2017-07-19 10:57:28 +00:00
Erik Bernhardson
7fd6ef9ca5 Support custom offsets from SearchResultSet
Work to support interleaved AB testing of search will display
interleaved results of two search configurations on the first page of
results, but standard results on all pages other than the first page.
This means that while 20 results may be requested, the next 'new' result
of the primary query may be at position 10. Allow the SearchResultSet
to declare what the new offset is.

Bug: T150032
Change-Id: I14c0c33249fcdb66f72f5966e2aa72781a34daee
2017-07-18 21:22:49 -07:00
Thiemo Mättig
d14faa6bed Remove auto-generated "Constructor" documentation on constructors
Having such comments is worse than not having them. They add zero
information. But you must read the text to understand there is
nothing you don't already know from the class and the method name.

Change-Id: I994d11e05f202b880390723e148d79c72cca29f0
2017-07-10 10:15:51 +00:00
jenkins-bot
e0f53bfb8d Merge "Fix phrase search" 2017-07-05 18:35:16 +00:00
David Causse
187ada1cf4 Fix phrase search
Partially revert I61dc536 that broke phrase search support.

Fix phrase search by making explicit that there are two
kind of legalSearchChars() usecases :

- the chars allowed to be part of the search query (including special
  syntax chars such as " and *). Used by SearchDatabase::filter() to
  cleanup the whole query string (the default).

- the chars allowed to be part of a search term (excluding special
  syntax chars) Used by search engine implementaions when parsing with
  a regex.

For future reference:
Originally this distinction was made "explicit" by calling directly
SearchEngine::legalSearchChars() during the parsing stage. This was
broken by Iaabc10c by enabling inheritance.
This patch adds a new optional param to legalSearchChars to make this
more explicit.

Also remove the function I introduced in I61dc536 (I wrongly assumed
that the disctinction made between legalSearchChars usecases was due
to a difference in behavior between indexing and searching).

Added more tests to prevent this from happening in the future.

Bug: T167798
Change-Id: Ibdc796bb2881a2ed8194099d8c9f491980010f0f
2017-07-03 11:44:48 +02:00
Umherirrender
b5cddfb27b Remove empty lines at begin of function, if, foreach, switch
Organize phpcs.xml a bit

Change-Id: Ifb767729b481b4b686e6d6444cf48b1f580cc478
2017-07-01 11:34:16 +00:00
jenkins-bot
0e25b05028 Merge "Add SearchIndexField::getEngineHints()" 2017-06-28 20:37:13 +00:00
jenkins-bot
01c3bf3431 Merge "Fix highlighting for phrase queries" 2017-06-27 23:44:27 +00:00
Umherirrender
be42e09aa8 build: Prepare for mediawiki/mediawiki-codesniffer to 0.9.0
The used phpcs has a bug, so the version 0.9.0 could not be enforced at the moment.
Will be fixed in next version, see T167168

Changed:
- Remove duplicate newline at end of file
- Add space between function and ( for closures
- and -> &&, or -> ||

Change-Id: I4172fb08861729bccd55aecbd07e029e2638d311
2017-06-26 17:14:31 +00:00
David Causse
f230f5dcc7 Fix highlighting for phrase queries
I think the bug was introduced during a cleanup in Iaabc10c.
I don't think that " should be part of the legalSearchChars at query
time, it seems to break the regex.
The strategy here is to distinguish legalSearchChars used query time vs
the ones used at index time by introducing:
SearchEngine::legalSearchCharsForUpdate()

Bug: T167798
Change-Id: I61dc53665e26d3c6c48caed78dd3bbde9a33def7
2017-06-26 09:53:13 +02:00
David Causse
b5aecfabd9 Add SearchIndexField::getEngineHints()
Allows search engine clients that implement custom definitions
to pass engine hints used at index time.
Hints are a way to fine tune the behavior of the search engine
when handling a particular field.
As of now this is introduced for CirrusSearch to let SearchIndexField
implementations to control how the noop script is configured.
The noop hint with CirrusSearch allows to (for example):
- ignore an update if a numeric value does not change for more than X%
- control the merge strategy of complex fields

Bug: T166589
Change-Id: Ia560e41d33013c30ac47e5a60543f8cb133e61fb
2017-06-22 16:03:01 +02:00
Paladox
54c56da85a Fix php code style
Preparation change for updating mediawiki code sniffer to 0.8.0

Change-Id: Ib0b3fe4afea9096ffa3a1347b4f7e07d3398b0b2
2017-05-05 12:03:54 +00:00
Brian Wolff
d4385537bc SECURITY: XSS in search if $wgAdvancedSearchHighlighting = true;
In the non-default configuration where $wgAdvancedSearchHighlighting
is set to true, there is an XSS vulnerability as HTML tags are
not properly escaped if the tag spans multiple search results

Issue introduced in abf726ea0 (MediaWiki 1.13 and above).

Bug: T144845
Change-Id: I2db7888d591b97f1a01bfd3b7567ce6f169874d3
2017-04-06 13:42:44 -07:00
Stanislav Malyshev
1e412aabdf Add deleted archive titles search
Allows search engine to suggest deleted titles for undelete search.
Note that the titles are still verified against the archive table,
to ensure search engine is not out-of-date.

Bug: T109561
Change-Id: Id6099fe9fbf18481068a6f0a329bbde0d218135f
2017-04-05 12:02:35 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Stanislav Malyshev
cae86036aa Allow fields to define their own merge strategy via callback.
Change-Id: Ic1cc1581f07381224d3f4fcba4feb1eac7085057
2017-03-07 10:36:53 -08:00
Aaron Schulz
e01fd44388 Move ResultWrapper subclasses to Rdbms
Change-Id: I6f3f0e85e268b24c57c537aa6ad8016e0b4cdddb
2017-03-03 00:44:41 +00:00
Bartosz Dziewoński
ecdef925bb Miscellaneous indentation tweaks
I was bored. What? Don't look at me that way.

I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.

Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
2017-02-27 19:23:54 +01:00
jenkins-bot
0a2297c406 Merge "Fixes for more robust dealing with content handlers." 2017-01-30 16:06:25 +00:00
David Causse
cc61a14751 Allow SearchEngine users to access features data
Useful in case the client wants to re-evaluate what was set
here, or if the SearchEngine implementation wants to expose
some of its states.
In our case it allows CirrusSearch to inform SpecialSearch
that we prefer to display search results with a new experimental
layout.

Bug: T156299
Change-Id: I7f661c852ef70ea7bc9ae2959f7d6e48776a9877
2017-01-27 15:12:10 +01:00
Stanislav Malyshev
60ffe51c79 Fixes for more robust dealing with content handlers.
Change-Id: I12a02da005f4b2bceaa850bd1f41a90ac4e1754a
2017-01-26 12:20:07 -08:00
Bartosz Dziewoński
0cccd68dc8 Code style: no space after unary minus operator
Searched for /([^\d\w\s\)\]]\s*)- \d/ to find potential issues.
It seems there's no PHPCS check for this, huh.

Also fixed typo in a comment in LoginSignupSpecialPage.

Change-Id: Iaab1a1f5a9f234971e550e7909aa5c3e0c02a983
2017-01-05 14:38:32 +01:00
Max Semenik
d4f3e554d7 Decrease the number of 'function says it should return something' errors
Change-Id: Ib5115fe5bbaa67d8a6e54cc3ba1ba7020e239e11
2016-12-15 16:05:52 -08:00
Erik Bernhardson
41f9c933b3 Fix two instances of calling function with too few args
At one point SearchIndexFieldDefinition was updated to require the
engine to be passed in, but it seems that update was missed here.

BackupDumper::loadPlugin() requires the second argument, set it to
the empty string to keep current behaviour.

Change-Id: Ifbd8fc4870ff63b2d338f8bb4d251d7a3477b989
2016-12-15 13:24:40 -08:00
Thiemo Mättig
00c3f09566 Remove empty lines from PHP and JavaScript comment blocks
This is a pure documentation change. It mostly removes empty lines from
comments (and entirely empty comments), as well as adds a few missing
documentation blocks and fixes a minor mistake. I hope it's ok to have
this in one patch. I can split it, please tell me.

Change-Id: I9668338602ac77b903ab6b02ff56bd52743c37c4
2016-12-09 09:01:06 +00:00
umherirrender
34fe90ac52 Remove empty lines at end of functions
It looks like there is something missing after the last statement
Also remove some other empty lines at begin of functions, ifs or loops
while at these files

Change-Id: Ib00b5cfd31ca4dcd0c32ce33754d3c80bae70641
2016-11-05 11:55:10 +01:00
dcausse
bd7df68603 Do not run exact db match if offset is > 0
When scrolling results on prefix search api the exact Title
match is always at pos 0 even if we want to scroll the results
by setting offset to > 0.

Change-Id: Ib02c9d3e479d739e6fe79014d962db50b6fd9de8
2016-09-27 15:28:14 +02:00
dcausse
5417b6edc7 Add new type SearchIndexField::INDEX_TYPE_SHORT_TEXT
Useful for short technical strings such as mime types.

Change-Id: If440378f2b7004abed87eca4bbde767212c4b062
2016-09-26 12:50:23 +02:00
jenkins-bot
82dcfbe20e Merge "Pass User to SearchEngine::getProfiles" 2016-09-20 20:30:36 +00:00
dcausse
16e2491a73 Pass User to SearchEngine::getProfiles
Useful for search engines that allow users to customize search profiles.

Depends-On: Icd577c8ebc6e162befe30bde4fe276e633d2e434
Change-Id: I471cd090730d2a25cb70d622ec3bebbe9583118c
2016-09-20 20:22:23 +00:00
dcausse
8b890bbe1b Extract replacePrefixes into a static method
Useful for some search engines that have a keyword that wants
to reuse this logic without building a new SearchEngine object.

Change-Id: Iee5bfd1da70b8339a98555ba062bd33b21f0b761
2016-09-19 16:54:06 +02:00
Aaron Schulz
d2202b8414 Add SearchEngineFactory::getSearchEngineClass() instead of using the DB
Change-Id: I46207cafec737cb830adc2bdf1b501a0d138b40d
2016-09-15 21:31:35 -07:00
Stanislav Malyshev
7e18cfc3b5 Infrastructure for augmenting search results
Bug: T117493
Change-Id: Ia5413a7846cc961026a2dc3542b619493bc76a23
2016-09-15 15:44:03 -07:00
jenkins-bot
dc178bf8f6 Merge "Clarify that $terms get preg_quoted by caller in SearchHighlighter" 2016-09-07 14:18:43 +00:00
Brian Wolff
6941283af4 Clarify that $terms get preg_quoted by caller in SearchHighlighter
Also fix misleading comment about which highlighter implementation
is default.

Change-Id: Ifb4d7fa79ec6325bdb0667fc10b6a1547b32ef69
2016-09-06 19:42:12 +00:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
jenkins-bot
fd8a5d4689 Merge "Remove SourceIndexField FLAG_SOURCE_DATA" 2016-09-01 18:59:55 +00:00
dcausse
2dc04ccdb4 Remove SourceIndexField FLAG_SOURCE_DATA
Change-Id: I080f06a5a09f2d67a153b491555d0dbf65c626d0
2016-09-01 17:03:56 +02:00
jenkins-bot
dc36560cdf Merge "Add DEFAULTSORT to search index field data" 2016-09-01 14:51:41 +00:00
dcausse
7c09f09432 Add DEFAULTSORT to search index field data
Added FLAG_SOURCE_DATA to support additional data that is not supposed to be
part of the default mapping.

Should merged with I1484c2e62788bedb57a42869a5fb25cd8f64482f, otherwize rebuilding
an index may add an extra field to CirrusSearch mapping.

Bug: T134978
Change-Id: Ia41f8eeb9dd4f764543bdd4d71b7a50de8101101
2016-08-29 16:51:57 +02:00
aude
64ee3d3269 Extract ParserOutput search index data fields from WikiTextContentHandler
Bug: T142491
Change-Id: I69b010b893135e53fac7f16f4b927b8fbcba06d2
2016-08-19 09:26:17 -04:00
Stanislav Malyshev
add1ebe2ab Make content handlers assemble content for search
Bug: T89733
Change-Id: Ie45de496ecc826211d98eea3a410c7639b4be0a4
2016-07-26 13:08:45 -07:00
Stanislav Malyshev
dc9e171e98 Improve comments on fields and fix opening_text - needs no highlights.
Change-Id: Iad6876aae109ad84c5534619f47c72edc900d704
2016-07-08 13:02:06 -07:00
Stanislav Malyshev
86b55ad03c Create API to allow content handlers to handle structured data definitions
Change-Id: Ia1738803c42f6114575587c1c838fec62b6f54aa
Bug: T89733
2016-07-06 13:41:20 -07:00
dcausse
31680aaddc Expose SearchEngine specific profiles
This patch introduces a way for SearchEngine implementations to expose
specific search profiles useful to fine-tune the various behaviors related to
search.

A SearchEngine can expose a list of profiles by overriding
SearchEngine::getProfiles( $profileType ), profileType stands for the type of
profile being customized. Two types are added in this patch:
- completion: exposed by ApiQueryPrefixSearch and ApiOpenSearch to control
  the behavior of the algorithm behind "search as you type" suggestions.
- fulltext query independent profiles: exposed by ApiQuerySearch to customize
  query indpendent ranking profiles (e.g. boost by templates/incoming
  links/popularity/...)

This patch allows api consumers that might have been confused by fuzzy
suggestions to switch to stricter profiles and to officialize the behavior
behind the hidden param cirrusUseCompletionSuggester. Or to control the
fulltext ranking behaviors like cirrusBoostLinks=(yes|no).

The list of profiles can be discovered by using ApiSandbox/ApiHelp and is totally
controlled by search engine implementations.

Bug: T132477
Change-Id: I66be724d8975976c98c91badbf421f237e014f89
2016-05-30 20:43:53 +02:00
dcausse
257c023666 Fix Undefined variable: namespaces in includes/search/SearchEngineConfig.php on line 109
Bug: T134305
Change-Id: I220886e12a6d083ac34a8a75bc77871e89dbf747
2016-05-03 21:41:15 +02:00
Ricordisamoa
e64035522d Fix and standardize Doxygen tags
* Use "@param datatype $paramname description" format

* String → string, Integer → int etc.

* @return $string → @return string

Change-Id: I860d222382cb4c5699d313b0600bd22503c8c385
2016-04-30 12:10:17 +02:00
Stanislav Malyshev
82c8c00ce2 Move wgContLang from config to injectable
Change-Id: Iffdc39f2de7d38ee9ef882bb796e8969e95e75c6
2016-04-27 11:55:17 -07:00
Stanislav Malyshev
34b02d87ac Convert SearchEngine to service containers
Change-Id: Icef1ecbed3d831557e0256fdfa53743b194007cc
2016-04-25 16:25:17 -07:00
Erik Bernhardson
5f26e87b79 search: getInterwikiResults returns array of SearchResultSets
This function was incorrectly annotated. We can clearly see from the
code that uses it, such as in SpecialSearch, that this is an array of
SearchResultSet objects.

Bug: T132625
Change-Id: I4af07d3c9a9b08fd1fa438ddb6b781f78472b26c
2016-04-14 09:04:12 -07:00
Erik Bernhardson
15483b22d7 Add a rewind function to SearchResultSet
This is implemented in CirrusSearch so the result set can be looped
through multiple times. In general having a rewind function on any
iterator is probably a good idea, so pushing it into the upstream
interface.

Change-Id: Iaac9c11d8742288610011a2a0f6282944d35d5f3
2016-04-13 14:15:31 -07:00
dcausse
6cbd56954e Fix Undefined index: 0 in SearchExactMatchRescorer.php on line 44
It seems to happen when opensearch or prefixsearch api is called with an
invalid namespace parameter.

Change-Id: I1dda12c93db03c520080e8ee57bd750396905a58
2016-03-31 16:26:37 +02:00
Erik Bernhardson
489fa6fca2 Allow titles with falsy title text in suggestions
A valid title, such as 0, would end up being ignored in this clause and
not providing a title. Make the empty string check stricter to allow
these titles through.

Change-Id: I938a7dfe294e979c19f720eadb6f96282b1c4588
2016-03-24 16:35:56 -07:00
dcausse
4c2f0e3940 completionSearch: try an exact match even if the backend returns no result
Unfortunately some backends (like Cirrus) are not able to do proper crossnamespace
lookup. If the backend returns no result we should still try an exact title match.

Bug:T129575
Change-Id: Ic08ec0aac0fd8a289f21753d74ae62ba509de841
2016-03-11 09:40:40 +01:00
Siebrand Mazeland
5b119a0e44 Replace uses of join() by implode()
All of core uses implode() consistently now.

Change-Id: Iba50898c64c43f356d1caf8869f484e90d9ff651
2016-03-08 18:24:16 +00:00
aude
3a5931c3e5 Fix comment in SearchEngine.php
Change-Id: Ib00b42d2210be5bc1125fa7ba74c27a5d7fbf36c
2016-03-01 10:48:55 +01:00
Bartosz Dziewoński
c161c46d26 Improve code suffering from PHP 5.3's lack of support for foo()[]
I searched for /\$(\S+) = (.+?\(.*?\);)\n.*?\$\1\[/, ignored
everything involving isset(), unset() or array assigments, then
skimmed through the remaining results and changed things where they
made sense. These changes were not automated, so please review them.

Change-Id: Ib37b4c66fc57648470f151ad412210b3629c2538
2016-02-28 22:49:20 +01:00
Kunal Mehta
b9668d6d80 build: Update mediawiki-codesniffer to 0.6.0, add "composer fix"
* Fix errors spotted by new release
* Introduce "composer fix", which uses phpcbf to automatically fix some
errors spotted by phpcs.
* Drop $PHPCS_ARGS variable that didn't work on Windows, and add -s flag
* Remove rules from phpcs.xml that are now in MW-CS ruleset.

Change-Id: I13e2155695918c918b67497ac65b85a03897095e
2016-02-17 02:54:42 -08:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Stanislav Malyshev
027972a20f Include completion search into SearchEngine
By default it still uses PrefixSearch and supports PrefixSearchBackend
but it can be deprecated and phased out and SearchEngine extensions used
instead.

New APIs:
- SearchEngine
	public function defaultPrefixSearch( $search );
	public function completionSearch( $search );
	public function completionSearchWithVariants( $search );

Search engines should override:
protected function completionSearchBackend( $search );

Bug: T121430
Change-Id: Ie78649591dff94d21b72fad8e4e5eab010a461df
2016-02-03 23:41:49 +00:00
nomoa
33e5c11e85 De-duplicate near match query terms when generating variants
zhwiki generates 9 variants: we should not run these queries if the term is unchanged.

Change-Id: If23d19761dea33bf4bdcf6495becc8e983915fde
2016-01-11 17:33:00 -08:00
Justin Du
3b0f55c492 Split SearchResultSet.php classes
Change autoload.php file to accomodate the split

Bug: T122637
Change-Id: I3a326aa329c3be64662825ac06b169ae96b8f53c
2016-01-03 17:12:10 -06:00
dcausse
353ae71963 Extracted some code from PrefixSearch for re-usability by other search engines.
(This patch is needed for Ida9b9f8)

Bug: T112028
Change-Id: I35aece88333a65f6b1f55f7a87e2d14de4f5bea7
2015-12-02 17:18:08 +00:00
umherirrender
7724e285fd phpcs: Fix some "Single space expected before elseif"
Found by new version of mediawiki/codesniffer
https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs/1939/consoleFull

Change-Id: I465bf0d1c89603b3dfc9867be3c0b1190829312d
2015-11-01 21:03:18 +01:00
Stanislav Malyshev
46a13941f9 Displaying search results for multiple wikis
Uses fully translated string like search-interwiki-results-abwiki
which should be present in WikimediaMessages.

Requires Ic4bdd9462f844febea2e402e4b89d9dcc4c836b6
Supports I0d0efacf3066b3b5fe0bfcb058dd0b0f9988ccae

Bug: T112349
Change-Id: Ib6524bc79e1648ccf6adc5745f0dbc4c26dcb0d2
2015-10-22 14:34:41 -07:00
Amir E. Aharoni
695ab56c47 Fix long line to make phpcs pass
Bug: T102614
Change-Id: I2b14938366a8cc0e911f98ebb0d3e31f0fd31899
2015-10-06 16:05:08 +03:00
jenkins-bot
1569779421 Merge "Update various @params from DatabaseBase to IDatabase" 2015-10-05 19:59:21 +00:00
Thiemo Mättig
56d2a644f7 Add null to @return tags if a method can return null
Change-Id: I420998351663d92c4a101f61842e40591eebcd5f
2015-10-05 17:53:13 +02:00
Aaron Schulz
0f7893f877 Update various @params from DatabaseBase to IDatabase
Change-Id: I98e44cdffb0fc0d729f69f702799139afb988c20
2015-10-05 05:24:29 +00:00
Siebrand Mazeland
a350af27a0 Fix a PHPCS warning in SearchMySQL.php
Change-Id: I2a9d91e24421bd4c514a97e00abc1a7d6f7b3cf1
2015-09-28 14:06:01 +02:00
Vivek Ghaisas
c54766586a Fix issues identified by SpaceBeforeSingleLineComment sniff
Change-Id: I048ccb1fa260e4b7152ca5f09b053defdd72d8f9
2015-09-26 23:06:52 +00:00
Amir E. Aharoni
c6f066b88c Make long lines shorter to pass phpcs in 6 files
Bug: T102614
Change-Id: I3159c33ddd16bded94a70f363258a294f407fc58
2015-09-26 17:07:06 +00:00
Erik Bernhardson
23acf6e70f Move query rewriting into search backend
Special:Search recently gained query rewriting behavior when the
original query returned no results.  We want to expose this query
rewriting behavior to both api and web requests.  Additionally we
want to be able to test different configurations of the query
suggestions.  This patch allows for both by moving the rewriting
from core into the search backend.

This defaults to enabled for Special:Search, and disabled for
ApiQuerySearch. Internal code that talks to the search backend
needs to specifically enable this feature.

Bug: T106888
Change-Id: I0a8f75759f9148f53358707369b8a7128215de86
2015-08-31 22:41:05 -07:00
jenkins-bot
f873b49965 Merge "PostgreSQL: Fix text search on moved pages" 2015-07-20 23:03:58 +00:00
umherirrender
1f2d67b5e8 Pass function name in SearchMySQL::minSearchLength
This allow to see the correct function name in the debug logs

Change-Id: Ic18120ead47d4647943473d4aa1c5c795669d2c0
2015-07-12 22:05:37 +02:00
Vivek Ghaisas
9f5b6f5aeb Fix whitespace issues around parentheses
Fix issues found by MediaWiki.WhiteSpace.SpaceyParenthesis sniff.

Bug: T102617
Change-Id: Iec7f71e64081659fba373ec20d9d2006306a98f4
2015-06-16 22:14:02 +03:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
Erik Bernhardson
c24303d6b0 Search: Allow searchContainedSyntax to be defined by constructor
The SearchResultSet is returned where no query is performed because it
is known ahead of time that no result exists. In cases where that is
known due to the syntax of the query (e.g. the syntax requires
searching a category that does not exist) this searchContainedSyntax
method returns the wrong value. This patch makes it possible for the
instantiating code to properly set the return value of this method.

Change-Id: I7b7be24f5630a48d2a561e4fda9cec696a8cacf9
2015-05-18 16:19:16 -07:00
Erik Bernhardson
28abb2620a Document namespaces member as nullable
This variable is set to null in the replacePrefixes method, document
it as such to prevent errors in consuming code.

Bug: T98082
Change-Id: I78880ffe590ed7193b8482c0ae41c8c69e495878
2015-05-05 11:11:32 -07:00
Kunal Mehta
fe928a0a7f SearchHighlighter: Stop checking for existence of "wfCite" function
…and instead look for the Cite class. Not really any better, but
it will let us eliminate the "wfCite" global function.

Change-Id: Icdf82cb9771e6ae9bcaa6a02629b1b11f840a5c6
2015-04-30 11:25:01 -07:00
Jeff
c1371e7f54 PostgreSQL: Fix text search on moved pages
When a page is updated under PostgreSQL, there is code to
de-index all but the most recent version of the page.  But
when a page is moved, it was accidentally de-indexing the
most recent version as well, because rev_text_id is not
incremented in that case.  A simple tweak to the SQL
fixes that.

I added code to the update script to find pages
previously corrupted by this problem and reindex them.

Bug: 66650
Change-Id: I52e1bbbd8592be5e7c7383c225e6b4c19bbe5b9e
2015-02-17 18:15:23 -08:00
Sam Reed
e6c1ab047f Remove copious amounts of commented out preg_replace calls
Change-Id: I9b9afea3dc6729598b240845c3d34357548cdd4a
2015-01-27 05:34:40 +00:00
Sam Reed
2dc1ca4ace Remove unused $fname temporary variables
Change-Id: Ia7a32ecdfc293e870e42cd7398fc017de0d59f11
2015-01-27 05:31:43 +00:00
Ricordisamoa
2ae155da52 Fix phpcs errors in includes/
Mostly Squiz.WhiteSpace.SuperfluousWhitespace.EmptyLines

Change-Id: I678b2f0902f11cd1dfa1611b9da24e7237df9122
2015-01-08 20:15:07 +01:00
jenkins-bot
96ca8551b2 Merge "Removed remaining profile calls" 2015-01-08 17:47:16 +00:00
jenkins-bot
152cd9e470 Merge "Add a sort parameter to SearchEngine" 2015-01-08 16:23:49 +00:00
Aaron Schulz
4ff8136807 Removed remaining profile calls
Change-Id: I31c81c78715048004fc8fca0f27d09c1fa71c118
2015-01-08 02:49:33 -08:00
Nik Everett
74b1c0d256 Add a sort parameter to SearchEngine
SearchEngine grows a method to list valid sort orders one to set the
sort for the next search, and one to read it.  The default implemenation
only supports 'relevance' and the documenation strongly urges that all
other implemenations leave that as the default.  Other implementations
can support other orders. Cirrus already supports title_asc and title_desc.

Change-Id: Ie946150c6796139201221dfa6f7750c210e97166
2015-01-07 13:41:18 -08:00
Chad Horohoe
aa21e125a3 Remove obvious function-level profiling
Xhprof generates this data now. Custom profiling of various
sub-function units are kept.

Calls to profiler represented about 3% of page execution
time on Special:BlankPage (1.5% in/out); after this change
it's down to about 0.98% of page execution time.

Change-Id: Id9a1dc9d8f80bbd52e42226b724a1e1213d07af7
2015-01-07 11:14:24 -08:00
Aaron Schulz
e369f66d00 Replace wfRunHooks calls with direct Hooks::run calls
* This avoids the overhead of an extra function call

Change-Id: I8ee996f237fd111873ab51965bded3d91e61e4dd
2014-12-10 12:26:59 -08:00
Brad Jorsch
28e37f55c9 Merge OpenSearchXml extension into core
There's really no reason for the extension to exist separately from
core, and merging it reduces the risks of bitrot in both the extension
(lots of deprecated functions there) and core (missing integration with
PageImages and TextExtracts, for example).

Change-Id: Ie0ab90902ede9499879402290006466efba479e9
2014-11-26 21:07:22 -08:00
jenkins-bot
2dfacba930 Merge "Allow search to return matching category" 2014-10-24 17:01:58 +00:00
Reedy
8e6fa108b8 or -> ||
Change-Id: Ic591f06f70c68bb2912b7f028f7f988eb658375d
2014-10-24 11:26:14 +01:00
Nik Everett
f682aced27 Allow search to return matching category
Will be used by Cirrus.

Change-Id: I60e6b19d3b59ebf9bb45cc61d16936dfc7e4c708
2014-10-23 20:15:18 -04:00
Kevin Israel
3daf61814d Remove redundant $db->resultObject() calls
DatabaseBase::query() already passes the value returned by the
database driver (usually an object or resource for SELECT queries)
through $db->resultObject(). There is no need to pass it through
the method a second time to get a ResultWrapper.

Change-Id: I5d34ab2bffb7f236d2f824b4ef0e8670f89d4599
2014-09-19 20:35:55 +00:00
Chad Horohoe
9b3f84e763 SearchResult: clean up construction
- Remove old pre-1.17 constructor format. No core or extension callers
- Remove only usage of newFromRow(), needless abstraction

Change-Id: Ifad161aa36837bb4c8b5c9aecac1c46e2e6cc84c
2014-09-12 12:39:44 -07:00
Chad Horohoe
a82024c637 Remove "related" searches
Poorly documented and never used anywhere properly.

- No core search backend supports it.
- MWSearch implements it but we've never turned it on.
- SolrStore copies MWSearch but it doesn't even work.
- CirrusSearch doesn't bother because it's silly.

Change-Id: I8b052e8e772030fba6c361fdb2775b10a7f7a15a
2014-09-12 09:28:05 -07:00
Chad Horohoe
3b7a003589 Reset searchTerms[] before populating.
Otherwise, multiple calls to searchQuery() would accumulate search
terms. searchTerms[] is defined in SearchEngine so we can't just get
rid of it altogether.

Change-Id: Iddbb3920e3a9d5849c99fe90d2d725d16019b51d
2014-09-05 06:51:27 -07:00
umherirrender
b0cfcd0fcb Add missing @return and @param to doc blocks
Change-Id: I9d99ba1968ed8f97624d957754c8847dfe1b41da
2014-08-27 21:57:45 +02:00
umherirrender
63dc5abc9a Fixed spacing
- Added space after reserved words: function, foreach, if
- Combined 'else if' into elseif
- Added braces to one-line statements
- Added spaces after comma, before parentheses

Change-Id: Ie5bbf680d6fbe0f0872dab2700c16b1394906a72
2014-08-27 18:31:50 +02:00
umherirrender
7c6a25856c Add missing @return to function docs
Change-Id: I45b9d02f94ecc58372268ec5e6a0b572a0b7e2a9
2014-08-23 23:14:57 +02:00
Stephan Gambke
e3a34617b2 Fix highlighting of results when the search result does not return termMatches
If the search engine does not know the concept of serch terms the search
result object will just return an empty array as defined in class
SearchResultSet.

In this case SearchHighlighter::highlightSimple will place a span between
each and every byte (yes, byte, not character, it will break multibyte
chars).

This patch will just output the first few lines of a page if no search
terms are available for highlighting.

Note: Highlighting the page name in the case of title matches is
questionable, IMHO. It might make more sense for this case as well to
just return the first few lines of the page.

Change-Id: I276418f271855fb99443188f51cc076289c6ba0d
2014-08-18 15:55:51 +00:00
umherirrender
c2de24efcd Add missing @param to function docs
Change-Id: Ib7ac94d05a04490f25dfd40b46b27973cbab582c
2014-08-14 19:38:57 +00:00
Chad Horohoe
af87cb4a3f Remove score display from search engine
Scores are an internal metric that should not be exposed to
users, plus most backends fail to even support it.

Removes PostgresSearch*-specific result classes as they're not
needed anymore.

Change-Id: I00acaabad0565b9a5b3524c992feea366eb74bcc
2014-08-07 14:19:37 +01:00
umherirrender
768ac15c70 Cleanup some docs (includes/[s-z])
- Swap "$variable type" to "type $variable"
- Added missing types
- Fixed spacing inside docs
- Makes beginning of @param/@return/@var/@throws in capital
- Changed some types to match the more common spelling

Change-Id: Ie419638e909a47aa72a274043604247830ee1a81
2014-07-24 19:43:44 +02:00
umherirrender
4ee680a8b3 Fixed spacing
- Removed spaces after not operator (!)
- Removed spaces inside array index
- use tab as indent instead of spaces
- Add newline at end of file
- Removed spaces after casts

Change-Id: I9ba17c4385fcb43d38998d45f89cf42952bc791b
2014-07-24 11:53:04 +02:00
Brian Wolff
8c953947ff rm $wgCountTotalSearchHits. It was broken.
If $wgCountTotalSearchHits was set to true, then the total
number of hits returned was zero, which caused Special:Search
to display no results.

I'm opting to remove the feature (although I don't have any
strong opinions about removal vs changing Special:Search), since
if someone both cares about performance and has a wiki where its
big enough to matter, they are going to need to use Cirrus anyways.

Change-Id: I1c3b908ae5423ce3dfbdc22b1a68dd81a85698aa
2014-07-08 03:41:13 -03:00
jenkins-bot
a30854859e Merge "Remove SearchEngineReplacePrefixesComplete hook" 2014-06-24 07:20:18 +00:00
Chad Horohoe
da0070e9d1 Remove SearchEngineReplacePrefixesComplete hook
This hook is poorly thought out. The only extension that uses it
can't possibly think it works how they're expecting.

Change-Id: I853a01afc8e922f22e949321a2f2343d264632a6
2014-06-23 08:48:45 -07:00
Chad Horohoe
cd9dc7ef3f Database search fixes:
- Move filter() function and make it protected, nothing uses it
  outside database-backed searching
- Use per-backend legal search characters rather than assuming the
  static implementation is right

Change-Id: Ic2b830b56137b2dfe68b9b9c3de012151e716952
2014-06-20 13:43:36 -07:00
jenkins-bot
a25d105a30 Merge "SearchResultSet: remove hasResults(), unused" 2014-06-20 20:15:02 +00:00
Chad Horohoe
9643a0a92d Remove SearchResultTooMany
Only worked for Postgres, and only worked halfway at that. Result
sets could be false for many reasons. No results, bad query, database
went to Mars. We shouldn't assume they're always "too many results."

Leave it up to the normal query error logging and move on.

Change-Id: Ieddd163e440ae54b152541d727c1afdbc4ab4fbd
2014-06-20 10:36:47 -07:00
Chad Horohoe
bea8e24db2 SearchResultSet: remove hasResults(), unused
SolrStore, MWSearch and CirrusSearch all implement this and are
now free to drop their implementations.

Change-Id: Ia04bd3fc8526c35c3c9590c7fe4e2db2bc120283
2014-06-20 07:24:09 -07:00
Kevin Israel
b43ac16fcc Remove use of strencode() outside Database classes
Change-Id: Idbc38d8089541e5154b2601297b429f7433fd59e
2014-06-02 13:45:26 -04:00
Nemo bis
5dc4dc099d Save advanced search namespace prefs on Special:Search itself
* Checkbox on own row below power search checkboxes per MatmaRex;
  avoiding a mw-search-ns* id leaves it untouched by All/None JS.
* The option searcheverything is removed: a "shortcut" which is no
  longer necessary now that options can be (un)selected at once
  with All/None buttons on search page itself.
* Require a token for saving: no accidental preferences changes.
* Keep the searchoptions/advancedsearchoptions prefs section in case
  something is using it (no known extension does though); options
  are converted to "api" type so it's empty and hidden by default.
* Add minimal documentation for saveSettings() and friends
  (@todo since 155ddf6de, 2009!).

Bug: 52817
Change-Id: I514cee835988600cc013658049e88a10b670e64a
2014-05-30 14:33:47 -07:00
Chad Horohoe
1cb1ac6278 Remove "Help/Project" profile from Special:Search
The vast majority of wikis have an empty help namespace and
searching project pages is mostly an advanced search feature.

Cluttering the UI with an extra tab that's (most likely) rarely
clicked isn't helping anyone.

Change-Id: I0c448bcee877e027ff95c96812aac86d89d4eb32
2014-05-12 11:10:18 -07:00
Siebrand Mazeland
b1aa60af25 Pass phpcs-strict on includes/search/
Change-Id: Ia38af607b1f2a8e49f60f1a659a14b801e7dcda0
2014-05-09 22:32:17 +02:00
Chad Horohoe
b826ba0bf3 Don't call legalSearchChars() statically so it can properly inherit
Change-Id: Iaabc10ce2905eefe9de8bceeec3290082ba9eb2e
2014-05-05 13:34:02 -07:00
Chad Horohoe
ff00b052ce Clean up database-backed SearchResultSets
SqlSearchResultSet basically handles all of the work in a DB-agnostic
manner. Remove specific implementations for MySQL, Mssql and Sqlite

Change-Id: Iae3fd5cc40dfbc50917be73d7ace668681e4148a
2014-04-22 12:56:36 -07:00
jenkins-bot
622f2ddb63 Merge "Fixed some @params documentation (includes/*)" 2014-04-22 10:05:49 +00:00
Jeff
96c7f7eaa6 PostgreSQL: Improve speed for page edit in imports
Whenever a new revision is added, a deferred update gets enqueued.
When it is fired, it clears the searchable text from all earlier
revisions for the article.  This becomes very slow for articles
with long revision histories, as it re-clears the textvector even
when it has already been cleared by earlier actions.  This leads to
very high load in the database for runs of importDump.php

This patch improves this situation by adding a condition to the WHERE
clause such that it does not update rows in which the textvector
is already NULL.

PostgreSQL cannot automatically remove such degenerate updates
in general because the updated rows must be locked and have their
transaction markers increased.  However, in this particular case
those things are unimportant.

This change improves the performance of importDump.php on a
wiki with long revision histories by 7 fold, and moves the
major bottleneck from the database to PHP.  It might also
improve the performance of ordinary page edits, but that
was not tested.

There are more improvements that could be made here.  For example,
a partial index or expression index could make it so that already
cleared rows do not have to be visited at all.  Or the deferred
update mechanism could have a notion of "idempotency" so that many
indentical updates enqueued during bulk loading would be condensed
to only a single execution.  However, this change is very much
simpler and is sufficient to shift the bottleneck elsewhere.

Change-Id: I458603767c6a86425010d02ffc1f8079c4b3c9a0
2014-04-20 21:31:07 -07:00
umherirrender
dcf6955e5c Fixed some @params documentation (includes/*)
Swapped some "$var type" to "type $var" or added missing types
before the $var. Changed some other types to match the more common
spelling. Makes beginning of some text in captial.

Change-Id: Ifbb1da2a6278b0bde2a6f6ce2e7bd383ee3fb28a
2014-04-20 23:33:05 +02:00
umherirrender
edc924869f Changed some comments from /// to /** @var */
Change-Id: I55b30099e4464c272c35bcabbddbb0b1364a5d65
2014-04-07 05:01:19 +00:00
jenkins-bot
e3825282d1 Merge "Remove "include redirects" option from search" 2014-03-26 16:45:30 +00:00
Chad Horohoe
7e8c48ac3d Remove "include redirects" option from search
Always include redirects. Search results are worse without them
and MWSearch has never respected this option anyway.

Change-Id: I0ebe321a3b14e7960aa98e3225dc5a19ba916e7d
2014-03-26 09:28:26 -07:00
Chad Horohoe
692257f959 Remove completely useless implementation of getInfo()
Embedding random strings in Special:Search's comments doesn't
help debugging and just increases page size.

Bug: 62768
Change-Id: I51270aa3f2cba921841e2d8ebbd4fa665542f8a9
2014-03-19 20:20:31 +00:00
Chad Horohoe
7fc1a5b137 Break up SearchEngine.php into a couple of other files
Change-Id: I753eec1d0cf78f2ea08897c7fd3b83f2c88be18d
2014-03-19 20:14:17 +00:00
umherirrender
7c314de876 Rename some local vars to start with a lowercase letter
Change-Id: I6e5975ed7351c1439eda19afaba5120c6afa50f1
2014-03-15 21:03:05 +00:00