Sometimes, we need to force an exact value and bypass the default
behavior of merging config variables.
This also renames setConfigValue to putConfigValue, to avoid confusion
about the behavior of that method.
Change-Id: I82c606632b94f974e655a44a0b63394de7a0804b
This allows a file other than LocalSettings.php to be used as the primary
settings file by setting the MW_CONFIG_FILE environment variable.
This also allows the primary settings file to use YAML or JSON format.
Using static configuration files should be the default in the future.
However, YAML files in the document root could easily be exposed to the
public. Better not to encourage that, and require them to be enabled
explicitly and loaded from a different place.
Bug: T294750
Change-Id: I7747f83481cb05a6d05f819be652259951183819
This reverts commit 4f7a4a2477.
Reason for revert: This change is good, just need some preparation in extensions.
Depends-On: I24221be2cedfa132fc94d39d72e4a133cc3cdb12
Depends-On: I5e6119b650e581c6aa5a1132aa071b49cff8b8ca
Change-Id: I5a5a9000751fa3914c9d432eb49475091b3bdb80
Maintenance scripts often need to manipulate configuration settings.
This introduces a way to do this cleanly via SettingsBuilder,
removing the need to rely on global variables.
Bug: T294739
Bug: T294742
Bug: T300128
Change-Id: Ibf443fd564bbbf388cce8ab4dabba55ebca0dfa4
This logic is not needed to run on every PHP process and was making
it difficult to run offline maintenance scripts without additional
complexity based on Maintenance::getDbType and DB_NONE.
Instead of skipping this only for DB_NONE, and establishing a pattern
that may spread to other ad-hoc places throughout the codebase, instead
remove this entirely from the eager set up code for all PHP processes
and move it to the service wiring and dependency injection.
That way, it naturally doesn't happen until and unless the DB service
is actually called upon. Scripts and entry point that need to disable
the DB service, can continue to use
MediaWikiServices::disableStorageBackend.
== Impact on SiteStatsUpdate ==
With wgCommandLineMode no longer being read at run-time from a global,
but in service wiring, this means SiteStatsUpdateTest can't change
the behaviour between CLI-like and Web-like, unless it e.g. resets
the 'DBLoadBalancerFactory' service first. Unfortunately, while most
any reset is supported, a reset of the 'DBLoadBalancerFactory' would
be unsupported as that would lose the temporary db clone context and
such, bringing us to either the developer's live db, or a broken set
up altogether. If there is a strong need for toggling oppertunistic
updates off and on at run-time, this could be supported in the
DeferredUpdates class perhaps, but we already have numerous methods
there (incl db begin/commit being a good proxy already), which this
test already used, so for now I've just removed the extra assertion
for this as it wasn't essential to that test.
Bug: T228895
Bug: T238436
Change-Id: Icf29bc484c155f52b6d8f61e5902233a15ba0c6d
Be strict about Maintenance classes that claim to not need database
access by disabling the storage backend. Any subsequent attempts to
access the backend will result in an error.
While this change may cause breakage where the DB_NONE claim is
currently violated, the strict enforcement should help with refactoring
existing maintenance classes to work in a completely offline environment
and with the creation of future offline maintenance classes.
Removed `getDbType()` implementation from `SqliteMaintenance` which
returned `DB_NONE` to suppress output of some CLI parameters while still
depending on database access.
Bug: T260827
Bug: T237148
Bug: T238436
Depends-On: If6fbb0c0de26a53836c061406e49a7448aecceb8
Change-Id: I57c47badfaedf5646853803ca4224bb8f15a026a
Maintenance scripts that are meant to operate without a database (those
that implement `getDbType()` and return `Maintenance::DB_NONE`) should
not call `setAgentAndTriggers` which results in allocation of a new DB
load balancer. Unconditionally calling this method makes it impossible
for scripts to run without a database or database configuration.
Bug: T260827
Bug: T237148
Bug: T238436
Change-Id: Ic69ea6f6ebda1ca6be5cbbd96c7a4a48adce34bd
Follow-up ba6490aa1e.
That patch worked around $executeContext iteration errors due to the
lack of recursion support in doUpdates()/handleUpdateQueue(). Errors
were triggered by MediaWiki::schedulePostSendJobs() enqueueing deferred
updates that invoke JobRunner::run(), which, in turn, invoke doUpdates()
for each job via JobRunner:: doExecuteJob(). The doUpdates() method was
changed to operate on the sub-queue for the in-progress DeferrableUpdate.
Further improve the recursion logic:
* Use classes for the scope stack and scope queues.
* Put *all* updates added during a DeferrableUpdate::doUpdate() call into
the subqueue, regardless of "defer until" stage or whether it implements
MergeableUpdate. Now, doUpdate() can run *everything* it enqueued instead
of a non-obvious subset. Note that the TransactionRoundDefiningUpdate
that invokes JobRunner was already a POSTSEND update before ba6490aa1eb;
the only effect of this change is that MergeableUpdate instances from jobs
will once again run in doExecuteJob().
* Make recursive DeferredUpdates::doUpdate() calls error out immediately
unless the DeferrableUpdate responsible is a TransactionRoundAwareUpdate
with the TRX_ROUND_ABSENT flag. This covers the schedulePostSendJobs()
scenario and only prohibits insane call patterns. Failing early avoids
the risk of handleUpdateQueue() dropping all the updates due to the same
DBTransactionError error in DeferredUpdates::attemptUpdate().
* Avoid recursion loop in tryOpportunisticExecute() when JobQueueDB is in
use and a large number of tasks are pending. This happened due to methods
like onTransactionPreCommitOrIdle() being used within JobQueueDB.
Mark DeferredUpdates::doUpdates()/tryOpportunisticExecute() as @internal
and create a Maintenance::shutdown() method to avoid a direct call in the
doMaintenance.php file.
Bug: T249069
Bug: T268840
Change-Id: Ib369f0e74243a48ababdb9cd83b155c9a0f5e741
Sniff was renamed, phpcs violations are already
suppressed in phpcs.xml, no longer need these
individual line suppressions
Change-Id: I92ca4c6d576f1f0abada103a218155cc3aae38dd
Per 4eb1e679d5, these are now documented as the bare minimum
entry point responsibilities. For WebStart these are already
consolidated, but for the CLI use case they were a bit scattered.
Consolidate these in the CLI entry point (doMaintenance.php)
for clarity.
* $IP was previously assigned in Maintenance::__construct,
which is called from doMaintenance.php.
It still executes at the same logical time, one line above
'new $maintClass()'.
* MEDIAWIKI was (for CLI) previously defined in Maintenance::setup()
which is called from doMaintenance.php.
This is now done a few lines earlier, and in the entry point file
instead.
For both of these, they are still available and set in all the same
scenarios as before.
Bug: T246076
Bug: T250003
Change-Id: I2a6f5e997a36502535eb383a95deac0ce74d83fb
After b873e929, when the CLI installation failed, the script will throw
a ServiceDisabledException.
This is because the installer disables DBLoadBalancerFactory service
during instantiation and throws the exception because the installation
failed to restore the service.
So I check if the service is enabled before try commit
the master changes.
Bug: T229601
Change-Id: Ia7589d14ee55bcb03a64856b6dd2c81d8bda783c
Don't mix them in with stdout (which might be redirected,
like dump output, making the errors hard to discover).
Change-Id: Ibed8c0e8dde3e44de60bf32abd3fc5ce5d29e1ba
T110209 caused maintenance scripts to fail on unknown parameters,
which is normally desirable. However, some extensions (notably
SemanticMediaWiki) need to support additional params and had no
way to add them to the list of expected parameters. It will
now be possible them to update.php via a new hook:
MaintenanceUpdateAddParams.
Bug: T213893
Change-Id: Ia40949ccb2f32090f21e0f3f7e5b9c4aef322330
PHP supports exception chaining. This patch will output the error
message and stack trace not just for the latest exceptions, but for all
exceptions in the chain, using the Exception::getPrevious() method. This
avoids the problem where aftereffects obscure the actual problem, because
only the last exception in the chain was printed.
Change-Id: If577c5c89bb3b3e5766400fff07d8cc0a2d82610
There's a whole lot of shutdown code in doMaintenance.php that is
skipped when you use exit() directly. In HHVM by default, destructors
are not even called.
There are a few cases where maintenance scripts want to do something after
doMaintenance.php returns, so returning null should continue to mean no exit.
Change-Id: I0891e2ee3af7ef2c64c03b70edcf9e281ce1e7ba
Clean up use of @codingStandardsIgnore
- @codingStandardsIgnoreFile -> phpcs:ignoreFile
- @codingStandardsIgnoreLine -> phpcs:ignore
- @codingStandardsIgnoreStart -> phpcs:disable
- @codingStandardsIgnoreEnd -> phpcs:enable
For phpcs:disable always the necessary sniffs are provided.
Some start/end pairs are changed to line ignore
Change-Id: I92ef235849bcc349c69e53504e664a155dd162c8
Follows-up 41ea7e2fef.
The following previously happened between PreConfigSetup and Setup
and must now happen either before it, after it, or moved inside it.
* WebStart: Detect missing composer.
This must be after Autoloader/Vendor but before the first call to
wfDebugLog (or other loggers) so that we can output a more descriptive
error instead of a generic "Unknown class" fatal error.
Moving it to before Setup is too early, and after is too late.
Move it to within Setup.php and make it work in CLI mode.
* WebStart: Install header callback
Moving it to before Setup is too early, and after is too late.
Move it to within Setup.php (no-op in CLI mode).
* WebStart/Maintenance: Load LocalSetings.
Must be between PreConfigSetup and Setup.
Move to Setup.php to maintain execution order.
Utilise MW_CONFIG_File for custom handling in Maintenance.php.
* WebStart: Initialise output buffering
Utilise (new) MW_SETUP_CALLBACK hook.
* WebStart: Display NoLocalSettings.php
Utilise MW_CONFIG_CALLBACK hook.
* Maintenance: Setting $wgLocalisationCacheConf, calling Maintenance::finalSetup.
Utilise (new) MW_SETUP_CALLBACK hook.
Change-Id: I633a6ff235b4275391c48034c0525d2fbfa3fecd
Introduce PreConfigSetup.php, which is common file-scope code run before
LocalSettings.php.
I'm not maintaining autoload.ide.php since it supports closed source
software which I don't have, and it apparently needs significant work to
make it not be weird and hacky.
Change-Id: I44ac69b6b00a51d015546b9766d89d1c59749334
* Make isTransactableQuery() exclude CREATE/ALTER.
Starting transactions for schema changes like this can cause
errors as it is not supported for MySQL and some Postgres
operations. Note that temporary tables are session-level,
so they are not effected by this change.
* Clean up the transaction logic in determineCoreSchema()
so a transaction is not left dangling.
* Fix broken getSchemaPath() call in PostgresInstaller.
* Avoid warnings in DatabasePostgres::closeConnection() if
mConn is already unset.
* Commit master changes in doMaintenance.php before running
deferred updates, just as MediaWiki.php does.
* Change E_WARNING to E_USER_WARNING to avoid notices in the
default /rdbms error handlers.
* Also avoid trying to rollback in MWExceptionHandler if the
LBFactory service is disabled, which just results in an error.
Bug: T147599
Change-Id: I64ccab7f9b74f60309ba0c9a8ce68337c42ffb0f
* Use OutputPage::output() as the method to mask latency, since it
takes a good while to run. By the time it runs, cache replication
should have caught up, so the reap call will likely not block.
* For redirects emitted after changes in POST, instead of masking
with OutputPage, add a parameter to the redirect and block on
the positions appearing. This uses the redirection RTT to mask
the replication latency.
Change-Id: Ib23690c302e8033610fef9a0ef451dafe8a5803e
This sets triggers on master position waits typically called
after commitMasterChanges() or in commitAndWaitForReplication().
Change-Id: I127a8fe3cfc319abfa84fcd221ee2dae191c6d3b
Maintenance scripts can be invoked regardless of whether an extension is
enabled on a wiki or not. On wiki farms where some wikis may have an
extension or not, this can potentially be rather confusing. Especially
when the script bails out with a fatal class missing error or a database
table missing.
This allows maintenance scripts to specify that they require an
extension by calling:
$this->requireExtension( 'ExtensionName' );
in the script's constructor.
Bug: T141531
Change-Id: Icfbf063bb9c9ac9e55c3a5a8ed815528a2c1ce1e
* All callers are either using commit already or would be fine
using it (e.g. Maintenance scripts and JobRunner that have
no real transaction open).
Change-Id: I9f54b27619da6dac2cb63d255995aabc4ee78002
It looks like a recent change converted this try/catch
statement from MWException to Exception, unfortunatly it
is still calling a method that only exists on MWException.
We arn't doing anything with the exception but exiting. Lets
just allow the exception to be uncaught and fail naturally. The
uncaught exception will still output an error and return
a non-0 exit code.
Change-Id: I3e0e9d283c255d2aba8139d675943e93d7cb021a
Introduces wfLoadExtension()/wfLoadSkin() which should be used in
LocalSettings.php rather than require-ing a PHP entry point.
Extensions and skins would add "extension.json" or "skin.json" files
in their root, which contains all the information typically
present in PHP entry point files (classes to autoload, special pages,
API modules, etc.) A full schema can be found at
docs/extension.schema.json, and a script to validate these to the
schema is provided. An additional script is provided to convert
typical PHP entry point files into their JSON equivalents.
The basic flow of loading an extension goes like:
* Get the ExtensionRegistry singleton instance
* ExtensionRegistry takes a filename, reads the file or tries
to get the parsed JSON from APC if possible.
* The JSON is run through a Processor instance,
which registers things with the appropriate
global settings.
* The output of the processor is cached in APC if possible.
* The extension/skin is marked as loaded in the
ExtensionRegistry and a callback function is executed
if one was specified.
For ideal performance, a batch loading method is also provided:
* The absolute path name to the JSON file is queued
in the ExtensionRegistry instance.
* When loadFromQueue() is called, it constructs a hash
unique to the members of the current queue, and sees
if the queue has been cached in APC. If not, it processes
each file individually, and combines the result of each
Processor into one giant array, which is cached in APC.
* The giant array then sets various global settings,
defines constants, and calls callbacks.
To invalidate the cached processed info, by default the mtime
of each JSON file is checked. However that can be slow if you
have a large number of extensions, so you can set $wgExtensionInfoMTime
to the mtime of one file, and `touch` it whenever you update
your extensions.
Change-Id: I7074b65d07c5c7d4e3f1fb0755d74a0b07ed4596
- Put Profiler, ProfileSection and TransactionProfiler in their own
files and rely on Autoloader to use them (maintenance has been
using the autoloader here for some time--we don't profile the
autoloader manually)
- This reduces overhead in WebStart/doMaintenance by only loading
three functions at profiler initialization and defers until the
first profiling call happens
- Inline callback functions in ProfilerSimpleText rather than having
public static functions.
- Small comment and code formatting changes in various touched files.
Change-Id: Idf27677c068c50b847152c523a33e7f0c33fdeeb
Shouldn't be needed and aren't for any core operations. If any
extension relies on these $wgTitles being set in maintenance
environments those extensions are broken and should be fixed.
Change-Id: Ie02a5042ab96e155d783d56d5340dd0da8e3d55c
This has been deprecated and mostly unused for quite some time.
Remove the few remaining uses that exist.
Change-Id: I522ef138d291cf7567250fef1de34bb41673bc6c
The Line continuation Coding conventions prefers the closing parenthesis
on the same line than the beginning curly braces. This is done for ifs
and functions.
Also move some boolean operator from the end of a line to the beginning
and changed some indentation to make the condition hopefully better
readable.
Change-Id: Id0437b06bde86eb5a75bc59eefa19e7edb624426
iPart of program to remove underscores from class names. Checked core and
600+ extensions for occurrences. All uses are in core, and are updated in
this change.
Change-Id: I432dc249d22053728013ae7d0d56c3c398021c5e
... as well as the require_once statements in includes/WebStart.php
and maintenance/doMaintenance.php, now that the autoloader lists
MWInit (since r85807 / c68957c5e3).
Also removed code paths in maintenance/userDupes.inc that seem to be
dead (class_exists( 'Revision' ) should always be true) and useless
global/require_once statements in languages/Language.php.
Follows-up Ic3e769f1fbad4f7ad26dd819406796fee48c6b45.
Change-Id: I48fd6810fdb923b3065ae98024912eb18d394415
These functions existed to work around a bug (fixed in PHP 5.3) and
a missing feature (added in PHP 5.2) in older versions of PHP;
therefore, they are no longer necessary.
Change-Id: Ifebbe3d449fc57fd83f8350c28f467605c1a07b7
