Commit graph

14 commits

Author SHA1 Message Date
daniel
bb10b7d528 Settings: add support for obsolete config
This allows config variables to be declared obsolete. Obsolete config
will be omitted from the schema, defaults, name constants, etc. The
purpose of keeping a declaration of obsolete config around is to allow
the updater to warn admins that they are using a config variable that no
longer has any effect, and provide them with a remedy.

The idea is that support for deprecated config can be removed after one
release per the stable interface policy, but the declaration of
obsolete config should be kept for as long as we support updates,
that is, at least two LTS releases.

See https://www.mediawiki.org/wiki/Topic:X4bh4nf3pe2ho5jj for
discussion.

Change-Id: Ia7a00742ea7a5311e820a6a43b11135a3f2a825f
2022-10-14 15:00:43 +02:00
Aryeh Gregor
b72b9a8c43 Move dynamic defaults into MainConfigSchema
The goal is to keep the actual default values for settings in the same
place as the setting is declared, and applied using the regular means
for loading the settings -- not in a separate piece of code that needs
to be loaded through some entirely different mechanism.

SetupDynamicConfig.php now contains a few categories of things:

* Post-processing of configuration settings, where already-set settings
  are altered. This could be moved to MainConfigSchema too as a separate
  set of methods.
* Processing of old aliases of settings (blacklist, slave) that are not
  registered as settings anymore and therefore are not available to
  MainConfigSchema. This could perhaps be moved to LocalSettings
  processing somehow?
* Setting $wgUseEnotif, which is also not registered as a setting.
  Easiest would be just to declare it as a setting and have it set
  unconditionally.
* Setting the actual timezone to $wgLocaltimezone. This is not related
  to configuration and should just be in Setup.php.

Bug: T305093
Change-Id: Ia5c23b52dbbfcb3d07ffcf5d3b7f2d7befba2a26
2022-07-07 09:55:48 +10:00
jenkins-bot
26577ae68c Merge "Support etcd as a source for SettingsLoader" 2022-06-30 16:52:51 +00:00
daniel
f2df03704e Add support for nested property schemas in MainConfigSchema.
This adds support for JSONSchema style property declarations with nested
schemas. This is a step towards using more nested structured for
configuration, rather than adding to the over 700 keys already defined
in the main config schema.

Defaults from property schemas are aggregated into a default value in
the top level schema. Descriptions are however not yet aggregated.

Change-Id: Iaf46a9ecc83bee3566098c56137a1be66bff2ab9
2022-06-29 16:34:43 +10:00
Dan Duvall
4f4c831111 Support etcd as a source for SettingsLoader
Added a EtcdSource to handle loading of settings from etcd. The
implementation is based on EtcdConfig but is much simpler due to
the reliance on the existing source caching layer.

GuzzleHttp\Client is used over MultiHttpClient as the latter depends on
MediaWikiServices and therefore should not be used during early
initialization.

A naive cache key is based on the etcd request URL, effectively
representing the etcd API version and settings directory, and uses
either the DNS SRV entry as the host name or the host name itself if
discovery is disabled.

The cache TTL is set to 10 seconds. The combination of this low TTL and
the naive key should replicate the current caching pattern of
EtcdConfig. Stale results are allowed for failover in case of temporary
unavailability.

At this time, the expiry weight was not changed from the suggested 1.0.
However, verification of this as a suitable early expiration coefficient
should be performed in a production like environment.

Bug: T296771
Change-Id: I782f4ee567a986fd23df1a84aec629e648a29066
2022-06-21 17:17:32 +00:00
Reedy
5f2584b648 tests: Rename fixtures/bad.json
Bug: T306524
Change-Id: I3b97c041bf2e1171bd6e31774de12ae1d73f95a4
2022-04-20 15:17:36 +01:00
daniel
123f4d8dfc Make MainConfigSchema more readable.
This allows for PHPDoc-style types to be used in MainConfigSchema,
and adds aliases for the 'object' and 'array' types, to avoid confusion.

The type 'map' can now be used for associative arrays, it will be
represented as 'object' in JSON Schema. Similarly, 'list' can be used to
refer to sequential arrays, it will be represented as 'array' in JSON schema.

In addition, the basic PHP types are mapped to their JSON counterparts,
including support for nullable types. For example:

"?map" will become [ "object", "null" ];
"float|false" will become [ "number", "boolean" ];

Change-Id: I89617d06afd1e97ee1bc9bc49b6c77bed7bd50d9
2022-04-07 12:57:51 +02:00
daniel
2fe23d6860 Use class constants to define config schema, rather than config-schema.yaml
Instead of maintaining the config schema as a yaml file, we
maintain it as a set of constants in a class. From the information in
these constants, we can generate a JSON schema (yaml) file, and an
php file containing optimized arrays for fast loading.

Advantages:
- PHP doc available to IDEs. The generated markdown file is no longer
  needed.
- Can use PHP constants when defining default values.

NOTE: needs backport to 1.38

Change-Id: I663c08b8a200644cbe7e5f65c20f1592a4f3974d
2022-03-17 21:20:03 +01:00
Dan Duvall
a824da0f34 Support stale cached settings for failover
Require a `CacheableSource` implementation to signal whether stale
settings may be served from the cache by implementing an
`allowsStaleLoad()` method. This will be useful for sources (e.g. etcd)
that expect some degree of unavailability and failover during
unavailability is desired.

In order to support stale results, `CachedSource` sets cache items with
an indefinite TTL but uses the computed expiry timestamp in the envelope
to consider whether the item is considered expired as opposed to a true
miss where the item is malformed or not present in the cache store.  If
a `SettingsBuilderException` is thrown during the source's `load()`
method but the cache check was not a real miss, the stale cached results
are returned.

Note there is currently no pruning of cache items so it is advised that
`CacheableSource` implementations for which stale loads are allowed also
implement an immutable `getHashKey()` based on constructor arguments.

Bug: T296771
Change-Id: Ida0698a237dc0939230799700b6f54956f033d50
2022-01-21 13:31:09 -08:00
Derick Alangi
db662e1db4
Follow up on I91cd04d14f0ff86125fc8870984dab045c2098b5
Fixed typo introduce when building PhpSettingsSource due to
copy-paste of boiler plate class code.

Change-Id: I013ebc5a3b041187a3aa312498140fa40c235fbf
2021-12-17 19:14:35 +01:00
Derick Alangi
6aac95eb44 Settings: Introduce PhpSettingsSource for provisioning settings
This is an intermediate step before converting the merge strategies or
other configs/config schemas to YAML.

Bug: T297320
Change-Id: I91cd04d14f0ff86125fc8870984dab045c2098b5
2021-12-09 20:35:32 +00:00
Dan Duvall
d83a7bcd09 Cache loading of SettingsBuilder sources
The `SettingsBuilder` now accepts a PSR-16 cache interface with which to
store and query settings before attempting to load from each source. By
default, no cache is used, but any object that implements the
`Psr\SimpleCache\CacheInterface` may be provided to the constructor.

An explicit dependency on "psr/simple-cache" has been added to
`composer.json`. Note that this dependency already existed in vendor
albeit it as a transitive one.

An APCu based `SharedMemoryCache` adapter is provided as a canonical
PSR-16 compliant interface for production use.

Sources are now queued by the `SettingsBuilder` when calling `load()`.
If a cache interface has been provided, and the source is considered
cacheable (implements `CacheableSource`), then it is wrapped as a
`CachedSource` which will query the cache first before loading from the
wrapped source.

Cache stampedes are mitigated using probabilistic early expiry. The
implementation for this was partially based on symfony/cache-contract
source code but also from the Wikipedia article and paper referenced
therein.

See https://en.wikipedia.org/wiki/Cache_stampede#Probabilistic_early_expiration

Bug: T294748
Change-Id: I52ab3899731546876ee58265bd4a1927886746dc
2021-11-29 12:54:59 -08:00
Petr Pchelko
120ef51cbf SettingsBuilder: Add YAML file format.
If php-yaml extension in installed, use that. Otherwise
we fallback to symfony Yaml parser.

php-yaml is about 20 times faster then symfony, for default-settings.yaml
it will take PHP-yaml 6ms to load it vs 100ms for symfony. But given
that the result will be cached, it's better not to bring in
a required native dependency.

Bug: T294751
Change-Id: I3ffde926c3f264cacf39810ff7bd338c9f78823d
2021-11-29 09:27:20 -08:00
Dan Duvall
9a4af25664 Introduced settings sources and formats
A `SettingsSource` is meant to represent any kind of local or remote
store from which settings can be read, be this a local file, remote URL,
database, etc. It is concerned with reading in (and possibly decoding)
settings data, and computing a consistent hash key that may be used in
caching.

A `SettingsFormat` is meant to detect supported file types and/or decode
source contents into settings arrays. As of now, JSON is the only
supported format but others may be implemented.

`FileSource` is the first source implementation, with its default format
being JSON, meant to read settings from local JSON files.

`ArraySource` is mostly useful for testing using array literals.

Refactored `SettingsBuilder` methods to use the new source abstractions.

Bug: T295499
Change-Id: If7869609c4ad1ccd0894d5ba358f885007168972
2021-11-15 14:07:59 -08:00