Commit graph

37 commits

Author SHA1 Message Date
Max Semenik
bdf7e3f5bd Set constant visibility, part 1
Change-Id: I3dad26b1a0bd469fa84fee5c15d9b581765ceb94
2019-10-18 02:19:24 +00:00
Umherirrender
f74400487f phan: Disable enable_class_alias_support
It is enabled for b/c in extensions, but not needed in core

Change-Id: I51dca12be9c77049f77563d9bf0edd07928c2300
2019-09-15 08:26:52 +00:00
Umherirrender
5a4d30ed09 Avoid Database::tableName in WikiExporter
Using * in select is not the prefered way.
List all needed columns to make the use visible and to avoid issues when
new fields gets added with big data.
As each column name is unique there is no need to get the table name for
prefixing the columns

The following columns no longer selected:
- log_user_text -> not used due to use of ActorMigration class
- log_actor -> Add by ActorMigration class
- log_comment_id -> Added by CommentStore
- log_page -> Unused in the writer, the ns/title pair is used instead

Move the arrays out of the loop, because there are not depending on
values changing in the loop

Change-Id: I140641b7ed75bc2b8db2e7612020d668f1be663b
2019-09-12 20:06:07 +02:00
Daimona Eaytoy
c659bc6308 Unsuppress another phan issue (part 7)
Bug: T231636
Depends-On: I2cd24e73726394e3200a570c45d5e86b6849bfa9
Depends-On: I4fa3e6aad872434ca397325ed7a83f94973661d0
Change-Id: Ie6233561de78457cae5e4e44e220feec2d1272d8
2019-09-03 17:19:21 +00:00
jenkins-bot
bd62b3562a Merge "Improve type hints in export related classes" 2019-07-25 21:42:32 +00:00
Derick Alangi
339211a1ea Avoid usage of deprecated Revision::* constants, use RevisionRecord
Change-Id: I872fc89e5c02dd6a3ae9cd7e76640b95dc33f514
2019-07-21 15:03:03 +01:00
Daimona Eaytoy
148b239f03 Add fields and docs to WikiExporter
Three fields were undeclared, thus raising some phan warnings.

Change-Id: Ib7934b507cb69d29a3d2422dadc24b12207a12ad
2019-07-09 10:33:03 +02:00
Umherirrender
f5fa7a94d9 Improve type hints in export related classes
Change-Id: I3a11173bc96611c69cdc615eba741c6e4f92824a
2019-07-05 18:41:56 +00:00
daniel
fdc3e9f952 Add support for xml dump schema 0.11
Bug: T174031
Change-Id: I2717019ea7efe36694bd2b2fba4dc2952a987cfc
2019-06-27 21:56:01 +00:00
daniel
dd14601afb Join slot and content tables when dumping XML
This introduces a way to construct a RevisionRecord based on a
known set of SlotRecords. To allow this to be used consistently
with the legacy revision schema, some tweaks had to be made
to getSlotsQueryInfo().

Bug: T220493
Change-Id: I5ea972bb07ca1cfb3a2ad8ef120aef77e460745c
2019-06-27 22:26:22 +02:00
Aaron Schulz
1fb1494c93 Use IResultWrapper in code comments instead of ResultWrapper
Change-Id: Idb813c20bef0d41d0f9f01440daab4fee6cdb38d
2019-06-22 17:58:39 +00:00
Derick Alangi
2dca5bbbf5 Remove unnecessary semi-colons
Change-Id: I9eb65bdfbd3aa581effc14ead801b9e89b0359c3
2019-06-12 14:35:59 +01:00
Ariel T. Glenn
c27ced4f31 always order by page_id for dumps of current revisions
Also drop ordering of revs within pages, since there is only one
revision being dumped

Bug: T207628
Change-Id: I5e4f0bea7b54506ca389818407c43152a290da6e
2019-05-27 18:42:54 +00:00
Ariel T. Glenn
7f51b9e040 allow xml page content or metadata dumps to target specific namespaces
We don't alter the db query for this, but throw away the extraneous
rows before doing any processing on them whatsoever.

Use of the DumpNamespaceFilter comes too late to avoid processing
for each revision done in XmlDumpWriter::writeRevision.

Bug: T220940
Change-Id: I9cb30ce612d862d97d96720ac68ff2327409f485
2019-04-18 14:42:50 +03:00
Ariel T. Glenn
804b7f1f0f for exports, make sure we compare page titles as strings only
...and not as numbers!! Also added strict compare for the namespaces
field while we're in here.

Bug: T220257
Change-Id: If68b79334188c2f3be5d254bea3c1e27d52c4a9f
2019-04-06 13:01:33 +03:00
daniel
45f3912bf1 Make the XML dump schema version configurable.
Bug: T174031
Change-Id: I979b6c8f0a72bc1f5ecce1d499d3fdfa0f671588
2019-03-21 12:43:32 +01:00
jenkins-bot
68b12dfded Merge "Make BackupDumper MCR compatible (main slot only)" 2019-03-20 02:15:43 +00:00
daniel
5988e35505 Make BackupDumper MCR compatible (main slot only)
This makes BackupDumper compatible with the new mechanism for accessing
revision content.

This requires some changes to the way database connections are re-used,
since RevisionStore/SqlBlobStore needs to be able to run queries against
the database while the overall result set is being streamed.

This change does not yet add handing for extra slots to BackupDumper.
That first needs a spec for how extra slots will be represented in the
XML schma (T174031).

NOTE: this changes the output of fetchText from using integer text_id
values to using content_address values (e.g. "tt:4567" for text row
with old_id 4567). It also changes fetchText to accept such addresses
as input, for forward-compatibility. XML stub dumps still use the
numeric format in the id attribute, pending T199121.

Bug: T198706
Change-Id: If4c31b7975b4d901afa8c194c10446c99e27eadf
2019-03-14 13:19:51 +00:00
Aaron Schulz
cb15755e92 Normalize use of "INNER JOIN" to "JOIN" in database queries
The ANSI SQL default join type is INNER and this might save
some line breaks here and there.

Change-Id: Ibd39976f46ca3f9b71190d3b60b76ca085787a00
2019-03-06 09:17:30 -08:00
Brian Wolff
a848eae679 Use htmlspecialchars() not htmlentities in xml export for validity
htmlentities() can output entity references that are invalid in XML.
Use htmlspecialchars() instead.

Additionally, cast user-id to int for phan-taint-check

Bug: T216348
Change-Id: Idf781f5a3ffc3c6463969b3f5af63f0f08ae837c
2019-02-17 11:23:50 +00:00
Bill Pirkle
94ec06e0bf Fix for missing end tag </page> on some exports
T203424 replaced streaming mode with batched queries.
However, it did not properly handle some values of
the $wgExportMaxHistory config variable, and emitted
broken XML. This change fixes that issue.

Bug: T207974
Change-Id: Iade3fc603e513da51b7a970c16275516c02ede49
2018-10-31 18:02:02 -05:00
Ariel T. Glenn
1113b1203c fix stubs dump query to use straight join
regression from a combination of
https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/380669/
and
https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/459885/

We have to do a straight join for all stubs, since they all
now order revisions by rev_id after deployment of the second
patchset above.

Bug: T207628
Change-Id: I4d2a311c14c66d4813eb9fc3c587fa3ddb958454
2018-10-23 18:33:02 +03:00
Bill Pirkle
085b6e4787 Replace WikiExporter streaming (unbuffered) mode with batched queries
WikiExporter allows streaming mode, using unbuffered mode on
the database connection. We are moving away from this technique.
Instead, do multiple normal queries and retrieve the
information in batches.

Bug: T203424
Change-Id: I582240b67c91a8be993a68c23831c9d86617350e
2018-09-28 10:55:05 -05:00
daniel
7c2d0202ab Allow dumps to function with MCR in read-new mode.
This is a temporary fix that forces the dump code to continue to
use the old database schema, even when MCR is in
SCHEMA_COMPAT_READ_NEW mode. This will continue to function until
SCHEMA_COMPAT_WRITE_OLD  mode is disabled.

Bug: T198561
Change-Id: Ic54ee703f47d1843f70fdb7185ac1b098f148680
2018-09-07 14:18:40 +02:00
Brad Jorsch
27c61fb1e9 Add actor table and code to start using it
Storing the user name or IP in every row in large tables like revision
and logging takes up space and makes operations on these tables slower.
This patch begins the process of moving those into one "actor" table
which other tables can reference with a single integer field.

A subsequent patch will remove the old columns.

Bug: T167246
Depends-On: I9293fd6e0f958d87e52965de925046f1bb8f8a50
Change-Id: I8d825eb02c69cc66d90bd41325133fd3f99f0226
2018-02-23 10:06:20 -08:00
addshore
e5879da149 Pass $key into CommentStore methods and use MediawikiServices
This allows CommentStore to be added to MediaWikiServices
without the need of an aditional Factory.

This change includes a compatability layer to allow the behaviour
from 1.30 to continue to be used while deprecated.

CommentStore::newKey has been deprecated.
Keys are now passed into the public methods of CommentStore
where needed.
The following CommentStore methods have had their signatures changed
to introduced a $key parameter, but when used in conjunction with
CommentStore::newKey behaviour will remain unchanged:
  * CommentStore::getFields
  * CommentStore::getJoin
  * CommentStore::getComment
  * CommentStore::getCommentLegacy
  * CommentStore::insert
  * CommentStore::insertWithTemplate

Change-Id: I3abb62a5cfb0dcd456da9f4eb35583476ae41cfb
2018-02-05 15:34:12 +00:00
Brad Jorsch
11cf01dd9a Add comment table and code to start using it
A subsequent patch will remove the old columns.

Bug: T166732
Change-Id: Ic3a434c061ed6e443ea072bc62dda09acbeeed7f
2017-08-30 15:05:00 +10:00
Umherirrender
718e63694d Add missing @param and @return documentation
Change-Id: I1d1098eec3933df6561cceef646576013ddc08c8
2017-08-11 22:17:01 +02:00
Umherirrender
a9007e8baf Add missing & to @param documentation to match functon call
Change-Id: I81e68310abcbc59964b22e0e74842d509f6b1fb9
2017-08-11 18:47:46 +02:00
Kunal Mehta
d1cf48a397 build: Update mediawiki/mediawiki-codesniffer to 0.10.1
And auto-fix all errors.

The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.

Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
2017-07-22 18:24:09 -07:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Aaron Schulz
e01fd44388 Move ResultWrapper subclasses to Rdbms
Change-Id: I6f3f0e85e268b24c57c537aa6ad8016e0b4cdddb
2017-03-03 00:44:41 +00:00
Bartosz Dziewoński
ecdef925bb Miscellaneous indentation tweaks
I was bored. What? Don't look at me that way.

I mostly targetted mixed tabs and spaces, but others were not spared.
Note that some of the whitespace changes are inside HTML output,
extended regexps or SQL snippets.

Change-Id: Ie206cc946459f6befcfc2d520e35ad3ea3c0f1e0
2017-02-27 19:23:54 +01:00
Ariel T. Glenn
327d8c8b54 add option to XML dump stubs of page ranges with explicit rev_id ordering
tested for stubs, text, logging with and without start/end values,
with and without orderrevs, seems to work as expected, with the
appropriate changes to the query.

Bug: T29112
Change-Id: I94ca4a06235bdbed384bb997deb7432bb5aaa5b9
2016-06-22 22:05:55 +03:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Reedy
e7719c215c Fix corrupt ©
Change-Id: I2a875cf770e19834fdd22d57691ed945e524d7d7
2015-12-31 01:01:23 +00:00
mhutti1
c92aa31aab Split Export.php classes into seperate files
Moved classes in Export.php to seperate files in the new directory
includes/export/ and updated autoload.php to these new locations.

Bug: T122531
Change-Id: Idd3bba5a85d65c952f2ff503bea2ca76624c9b7f
2015-12-28 22:51:06 +00:00