Commit graph

212 commits

Author SHA1 Message Date
Kunal Mehta
d1cf48a397 build: Update mediawiki/mediawiki-codesniffer to 0.10.1
And auto-fix all errors.

The `<exclude-pattern>` stanzas are now included in the default ruleset
and don't need to be repeated.

Change-Id: I928af549dc88ac2c6cb82058f64c7c7f3111598a
2017-07-22 18:24:09 -07:00
Aaron Schulz
dd359741cc Move DB errors to Rdbms namespace
Change-Id: I463bd86123501abc68fdb78b4cda6110f7af2549
2017-04-15 10:47:41 -07:00
Aaron Schulz
21e71e0235 Use IDatabase type hints in /maintenance
Relatedly, move lockTables()/unlockTables() to IMaintainableDatabase

Change-Id: Ib53e9fa948deb2f9a70f0ce16c002613d0060bf9
2017-04-07 23:37:41 +00:00
Aaron Schulz
488a647831 Move IDatabase/IMaintainableDatabase to Rdbms namespace
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
2017-03-28 15:32:38 -07:00
Aaron Schulz
4a177b34ef Move LBFactory to Rdbms namespace
Change-Id: I5ae10783228d0252284807c9562bc8e328d4becb
2017-02-03 17:24:03 -08:00
addshore
b12086b618 Get ConfigFactory & MainConfig from MediaWikiServices
Change-Id: Iafdd7e00747060572463ffb05aae4543f3a06163
2016-11-23 00:12:38 +00:00
umherirrender
34fe90ac52 Remove empty lines at end of functions
It looks like there is something missing after the last statement
Also remove some other empty lines at begin of functions, ifs or loops
while at these files

Change-Id: Ib00b5cfd31ca4dcd0c32ce33754d3c80bae70641
2016-11-05 11:55:10 +01:00
Tim Starling
d831844364 Parser test file editor
Add editTests.php, which provides an interactive interface to the
parser tests, with semi-automated editing.

Change-Id: I1a20d007ba4627d562a16c03849bbad7aec0e516
2016-10-28 16:15:59 +11:00
Aaron Schulz
d1215eb7a5 Small DB classes IDEA warning fixes and cleanups
Change-Id: I65b5f452a364b329bb671540f4115dbc4f40eebd
2016-10-23 22:03:37 -07:00
Aaron Schulz
30f4b3c103 Replace DatabaseBase => Database in more places
Change-Id: If37a7909056bf2c31a8228cbc84f0fbbf5f1c517
2016-09-28 15:53:02 -07:00
Aaron Schulz
3a839c5927 Inject schema vars into DatabaseMysql
Also fix broken patchSql.php maintenance script

Change-Id: I45bccb0e2e10bd06651a551fa08ed0c66d11eb6a
2016-09-22 21:07:27 +00:00
Aaron Schulz
16b4e3a9f1 Avoid global state in DatabaseBase::factory()/query()
Change-Id: Ibb4f1c0dafea071a1c34e0cd5b5c15b8b4bb7bc6
2016-09-16 00:40:57 +00:00
Aaron Schulz
0e5cd18b74 Rename LBFactory => LBFactoryMW and make LBFactory in /libs
The former extends the later with MW-specific logic.

Also removed a wf* method call from ChronologyProtector.

Change-Id: I325f59b7467ab9c2137731d1ce69816f5a020f03
2016-09-15 17:14:32 +00:00
Tim Starling
6117fb244f Refactor parser tests
Merge the PHPUnit parser test runner with the old parserTests.inc,
taking the good bits of both. Reviewed, pared down and documented the
setup code. parserTests.php is now a frontend to a fully featured
parser test system, with lots of developer options, whereas PHPUnit
provides a simpler interface with increased isolation between test
cases.

Performance of both frontends is much improved, perhaps 2x faster for
parserTests.php and 10x faster for PHPUnit.

General:

* Split out the pre-Setup.php global variable configuration from
  phpunit.php into a new class called TestSetup, also called it from
  parserTests.php.
* Factored out the setup of TestsAutoLoader into a static method in
  Maintenance.
* In Setup.php improved "caches" debug output.

PHPUnit frontend:

* Delete the entire contents of NewParserTest and replace it with a
  small wrapper around ParserTestRunner. It doesn't inherit from
  MediaWikiTestCase anymore since integrating the setup code was an
  unnecessary complication.
* Rename MediaWikiParserTest to ParserTestTopLevelSuite and made it an
  instantiable TestSuite class instead of just a static method. Got rid
  of the eval(), just construct TestCase objects directly with a
  specified name, it works just as well.
* Introduce ParserTestFileSuite for per-file setup.
* Remove parser-related options from phpunit.php, since we don't
  support them anymore. Note that --filter now works just as well as
  --regex used to.
* Add CoreParserTestSuite, equivalent to ExtensionsParserTestSuite,
  for clarity.
* Make it possible to call MediaWikiTestCase::setupTestDB() more than
  once, as is implied by the documentation.

parserTests.php frontend:

* Made parserTests.php into a Maintenance subclass, moved CLI-specific
  code to it.
* Renamed ParserTest to ParserTestRunner, this is now the generic
  backend.
* Add --upload-dir option which sets up an FSFileBackend, similar
  to the old default behaviour

Test file reading and interpretation:

* Rename TestFileIterator to TestFileReader, and make it read and buffer
  an entire file, instead of iterating.
* The previous code had an associative array representation of test
  specifications. Used this form more widely to pass around test data.
* Remove the idea of !!hooks copying hooks from $wgParser, this is
  unnecessary now that all extensions use ParserFirstCallInit. Resurrect
  an old interpretation of the feature which was accidentally broken: if
  a named hook does not exist, skip all tests in the file.
* Got rid of the "subtest" idea for tidy variants, instead use a
  human-readable description that appears in the output.
* When all tests in a file are filtered or skipped, don't create the
  articles in them. This greatly speeds up execution time when --regex
  matches a small number of tests. It may possibly break extensions, but
  they would have been randomly broken anyway since there is no
  guarantee of test file execution order.
* Remove integrated testing of OutputPage::addCategoryLinks() category
  link formatting, life is complicated enough already. It can go in
  OutputPageTest if that's a thing we really need.

Result recording and display:

* Make TestRecorder into a generic plugin interface for progress output
  etc., which needs to be abstracted for PHPUnit integration.
* Introduce MultiTestRecorder for recorder chaining, instead of using
  a long inheritance chain. All test recorders now directly inherit from
  TestRecorder.
* Move all console-related code to the new ParserTestPrinter.
* Introduce PhpunitTestRecorder, which is the recorder for the PHPUnit
  frontend. Most events are ignored since they are never emitted in the
  PHPUnit frontend, which does not call runTests().
* Put more information into ParserTestResult and use it more often.

Setup and teardown:

* Introduce a new API for setup/teardown where setup functions return a
  ScopedCallback object which automatically performs the corresponding
  teardown when it goes out of scope.
* Rename setUp() to staticSetup(), rewrite. There was a lot of cruft in
  here which was simply copied from Setup.php without review, and had
  nothing to do with parser tests.
* Rename setupGlobals() to perTestSetup(), mostly rewrite. For
  performance, give staticSetup() precedence in cases where they were
  both setting up the same thing.
* In support of merged setup code, allow Hooks::clear() to be called
  from parserTests.php.
* Remove wgFileExtensions -- it is only used by UploadBase which we
  don't call.
* Remove wgUseImageResize -- superseded by MockMediaHandlerFactory which
  I imported from NewParserTest.
* Import MockFileBackend from NewParserTest. But instead of
  customising the configuration globals, I injected services.
* Remove thumbnail deletion from upload teardown. This makes glob
  handling as in the old parserTests.php unnecessary.
* Remove math file from upload teardown, math is actually an extension
  now! Also, the relevant parser tests were removed from the Math
  extension two years ago in favour of unit tests.
* Make addArticle() private, and introduce addArticles() instead, which
  allows setup/teardown to be done once for each batch of articles
  instead of every time.
* Remove $wgNamespaceAliases and $wgNamespaceProtection setup. These were
  copied in from Setup.php in 2010, and are redundant since we do
  actually run Setup.php.
* Use NullLockManager, don't set up a temporary directory just for
  this alone.

Fuzz tests:

* Use the new TestSetup class.
* Updated for ParserTestRunner interface change.
* Remove some obsolete references to fuzz tests from the two frontends
  where they used to reside.

Bug: T41473
Change-Id: Ia8e17008cb9d9b62ce5645e15a41a3b402f4026a
2016-09-12 16:11:42 +10:00
jenkins-bot
ba67060734 Merge "Make Maintenance::commitTransaction() avoid using deprecated method" 2016-09-06 06:16:42 +00:00
Aaron Schulz
950cf6016c Rename DB_SLAVE constant to DB_REPLICA
This is more consistent with LoadBalancer, modern, and inclusive
of master/master mysql, NDB cluster, and MariaDB galera cluster.

The old constant is an alias now.

Change-Id: I0b37299ecb439cc446ffbe8c341365d1eef45849
2016-09-05 22:55:53 -07:00
Aaron Schulz
583fd0b623 Make Maintenance::commitTransaction() avoid using deprecated method
Change-Id: I0bbbdf492265f05ede4864aca0b4df15c3e83754
2016-09-05 18:46:11 -07:00
jenkins-bot
a206443c0e Merge "Change "slave" => "replica DB" in /maintenance" 2016-09-06 01:05:50 +00:00
Aaron Schulz
c0a9ab0f6d Change "slave" => "replica DB" in /maintenance
Change-Id: Ibd3d617901130378a935402326cd4eefbb382c9e
2016-09-06 00:13:08 +00:00
Aaron Schulz
0622fbb077 Set "run when DBs idle" DeferredUpdates logic in Maintenance
* This is a more robust version of the one inside
  DeferredUpdates (checking all DBs), which will be 
  replaced in a later commit.
* Make sure the listener is restored when the LB gets
  torn down in tests. Also, it should respect the
  $wgCommandLineMode value as tests can change it.

Change-Id: I1e3faa5a058df44e2d27ab5ac185930867eb68ac
2016-09-01 22:09:21 +00:00
Aaron Schulz
44f486ffac Use transaction listener to run DeferredUpdates in CLI mode
This sets triggers on master position waits typically called
after commitMasterChanges() or in commitAndWaitForReplication().

Change-Id: I127a8fe3cfc319abfa84fcd221ee2dae191c6d3b
2016-09-01 03:30:00 +00:00
Tim Starling
535501ed62 Add a script to build an HHVM RepoAuthoritative bytecode file
This script compiles a list of known MW sources, including core and
extensions, and generates a RepoAuthoritative bytecode file. Some ideas
were taken from Bryan Davis's scap-hhvm-compile shell script.

Also, renamed the old maintenance/hiphop directory to maintenance/hhvm,
respecting the rename of the upstream project.

Change-Id: I55798729d0553d2840e8099e81d604a814e8664c
2016-08-16 13:32:19 +10:00
Kunal Mehta
4de667f3c2 maintenance: Allow having a nicer error message if an extension isn't enabled
Maintenance scripts can be invoked regardless of whether an extension is
enabled on a wiki or not. On wiki farms where some wikis may have an
extension or not, this can potentially be rather confusing. Especially
when the script bails out with a fatal class missing error or a database
table missing.

This allows maintenance scripts to specify that they require an
extension by calling:
 $this->requireExtension( 'ExtensionName' );
in the script's constructor.

Bug: T141531
Change-Id: Icfbf063bb9c9ac9e55c3a5a8ed815528a2c1ce1e
2016-07-30 10:38:51 +00:00
Kevin Israel
e257d0313f Maintenance: Use __DIR__ instead of dirname( __FILE__ )
dirname( __FILE__ ) was used because it worked in PHP 5.2, allowing
the version check to function correctly. However, Maintenance.php
now uses the short array syntax, which is a parse error before 5.4:

> Parse error: syntax error, unexpected '[' in maintenance/Maintenance.php
> on line 66

Also, do the same in install.php where it includes Maintenance.php.

Change-Id: I3373a02c684bb6cbde83d7bedc204818f3394602
2016-06-14 20:44:30 -04:00
Tim Starling
23361af7fd Respect undeclared command line options
This was broken by I847d45684ccd. We should really give an error when an
undeclared option is given (not ignore the option as in the previous
code), but most commandLine.inc users rely on undeclared options being
passed through to the $options global, and the change in I847d45684ccd
was apparently accidental.

Change-Id: Icec654a386bc79bf3e4ba81644c3c51ac5e093ba
2016-03-10 16:22:21 +11:00
Kunal Mehta
6e9b4f0e9c Convert all array() syntax to []
Per wikitech-l consensus:
 https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html

Notes:
* Disabled CallTimePassByReference due to false positives (T127163)

Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
2016-02-17 01:33:00 -08:00
Max Semenik
59db24e90b Use addDescription() instead of accessing mDescription directly
Change-Id: I0e2aa83024b8abf5298cfea4b21bf45722ad3103
2016-01-30 01:28:32 -08:00
Aaron Schulz
ff1c845082 Make Maintenance::commitTransaction() do slave wait checks
This makes it less likely to be forgotten in scripts.

Change-Id: I9b939dcda8d6191a2a5ee3b3b50d2ec950dc0402
2015-12-31 15:30:16 -08:00
Andrew H
1ae06b4f3e Add loadWithArgv() to Maintenance class
Very useful for passing in arguments to test Maintenance scripts.
Also, add a comment clarifying when $orderedOptions is available.

Change-Id: Ib25b3b36816bdf566c427b67646554a31a9fef0f
2015-12-31 01:18:48 +00:00
Aaron Schulz
fa8e1a9b00 Clean up transactions in maintenance scripts
Add transaction methods to complement getDB().
This makes it easy to grep for direct begin()/commit()
calls to IDatabase by having script use their own
wrapper. Maintenance scripts are one of the few places
that can (and need to) use begin/commit instead of the
start/end atomic methods.

Eventually, there should be almost no direct callers
and those methods can be made stricter about throwing
errors on nested calls.

Change-Id: Ibbfc7a77c0d2a55f7fc2261087f6c3a19061e0aa
2015-12-30 23:40:35 +00:00
Andrew H
0a0b02b56c Add support for specifying options multiple times in Maintenance scripts.
Bug: T122588
Change-Id: I847d45684ccd4054f4a159394266dc3e5506bbdb
2015-12-30 04:33:29 +00:00
umherirrender
04bef3dd66 phpcs: Use __DIR__ constant instead of calling
Replaced all dirname(__FILE__) by __DIR__ or added
@codingStandardsIgnore

Found by tests:
https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs/2736/consoleFull

Change-Id: I90ff10f183ed60175fe580c43d73c0e57fd04234
2015-11-15 21:14:42 +01:00
Amir E. Aharoni
34a07796c1 Make lines short to pass phpcs in files under maintenance/
Bug: T102614
Change-Id: Ieab3773a9ff8328e3672478a82ab31d8710fa8e2
2015-09-27 15:48:56 +00:00
Vivek Ghaisas
c54766586a Fix issues identified by SpaceBeforeSingleLineComment sniff
Change-Id: I048ccb1fa260e4b7152ca5f09b053defdd72d8f9
2015-09-26 23:06:52 +00:00
Kunal Mehta
f6e5079a69 Use mediawiki/at-ease library for suppressing warnings
wfSuppressWarnings() and wfRestoreWarnings() were split out into a
separate library. All usages in core were replaced with the new
functions, and the wf* global functions are marked as deprecated.

Additionally, some uses of @ were replaced due to composer's autoloader
being loaded even earlier.

Ie1234f8c12693408de9b94bf6f84480a90bd4f8e adds the library to
mediawiki/vendor.

Bug: T100923
Change-Id: I5c35079a0a656180852be0ae6b1262d40f6534c4
2015-06-11 18:49:29 +00:00
aude
9880d5b60b Check for dependencies in entry point and Maintenance.php
This way, if someone tries to install MediaWiki via
either web installer or install.php maintenance script,
after obtaining from git, they get some useful information
on what to do.

Put the checks alongside the php version check, as
stuff installed via composer (mustache / lightncandy)
is used for NoLocalSettings.php and install.php
attempts to use logging stuff.

Also tried to make PHPVersionError look slightly nicer,
with some <h2> elements and more padding for the <p>
elements.

and centralized this code in one place, as much as possible,
for improved ease of maintenance.

Bug: T90438
Bug: T88951
Change-Id: Iae4eb42c4266dbe9213c5de8a96fccfbeaa9acb0
2015-05-23 15:05:11 +00:00
Aaron Schulz
929b280b99 Unbreak --profile=text for CLI scrips
* Fixes "Undefined index: threshold" and
  "Invalid argument supplied for foreach()" errors
  so that output actually can show

Change-Id: Ibf8090f265d988c58d1752319336a5146a28f4d0
2015-05-08 19:09:31 -07:00
Ori Livneh
03d941ed42 Only configure the transaction profiler if profiling is enabled
Follows I0fe14bcda.

Change-Id: I4eb77d3b5ead587da6f01dec00fa6c5852225fcc
2015-05-06 17:45:48 -07:00
Aaron Schulz
7ea13643f5 Added $wgTrxProfilerLimits and slow query limits
* Limits are now configurable instead of being hard-coded

Change-Id: I99133586eb82e8e9e84061548c8d1a99695fde5c
2015-04-28 10:18:11 +02:00
jenkins-bot
b8f78c331c Merge "Add page_restrictions to readlock in lockSearchindex" 2015-04-12 18:32:19 +00:00
Aaron Schulz
ea5f44587a Log huge write queries in CLI scripts
* This is already done for web scripts, but with lower limits

Bug: T95382
Change-Id: I0fe14bcdaebc3568b0cb7af8084b7bfefeb82826
2015-04-08 07:06:03 +00:00
daniel
145f1c1bb4 Allow a lonely "-" as positional argument
Change-Id: I658988c8f955103d6bc4bd1047819a37de0d505d
2015-03-02 02:18:54 +00:00
Physikerwelt
f50de5d93d Add page_restrictions to readlock in lockSearchindex
Bug: T86156
Change-Id: Ie003dd279e2b157a628e415e39f231d5cb1ab468
2015-02-24 22:38:17 +00:00
Tim Starling
97a802407e Workaround for HHVM bug in readline()
https://github.com/facebook/hhvm/issues/4776

Change-Id: I9d3e6817cee2fa2c11e02989a4b11915f0c708f5
2015-02-10 06:13:15 +00:00
Aaron Schulz
acb4b73073 Avoid fatal error with profiler in Maintenance.php
Error Message: "Fatal error: Class name must be a valid object or a string"

Change-Id: Ia21f4d2c45aaaa12a270dd0fda7274008736a899
2015-01-16 19:21:28 +00:00
Chad Horohoe
24bc5fbec2 Remove silly reference ampersands from database functions in maint
This is 2015 and we use PHP5 I think.

Change-Id: I012de0df540fc91577585de939bb929c68664222
2015-01-05 11:32:44 -08:00
Ricordisamoa
12dec5d85d Fix some stuttering in comments and documentation
Change-Id: I9c0088b9aab37335203cad45a1d6fa8ac3f43321
2014-12-17 19:44:10 +00:00
Ricordisamoa
fc5fd5c37a Typo fixes and non-code tweaks
Skipped replacements:
* prefered → preferred
* prolly → probably

Skipped files:
* resources/lib/jquery.ui/jquery.ui.mouse.js
* resources/lib/jquery/jquery.form.js

Change-Id: Ib7923f362ddfca1b892bf5d601785d6b5aa5d44c
2014-12-12 18:31:15 +00:00
Mark A. Hershberger
f0723166f1 Fix type hint to match php docs
From http://php.net/manual/en/function.posix-isatty.php

bool posix_isatty ( mixed $fd )

Determines if the file descriptor fd refers to a valid terminal type device.

fd

    The file descriptor, which is expected to be either a file
    resource or an integer. An integer will be assumed to be a file
    descriptor that can be passed directly to the underlying system
    call.

    In almost all cases, you will want to provide a file resource.

Change-Id: Ief19ac1f09e0a955c8f5d5af28b87b8e8d427a2e
2014-12-10 20:37:47 +00:00
Aaron Schulz
66170e7e59 Adding basic profiler sampling support and restored the --profiler script option
* Also enforce that the profiler is normally off in CLI mode

Change-Id: I35faedff818af2ad459b544c9ad50e77b54b378e
2014-11-21 15:14:18 -08:00