wiki.techinc.nl/tests/selenium/README.md

74 lines
2.2 KiB
Markdown
Raw Normal View History

Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
# Selenium tests
## Getting started
See <https://www.mediawiki.org/wiki/Selenium> for how to best
run these locally. Below the internal prerequisites are documented,
but you might not need to install these yourself.
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
## Prerequisites
- [Chromium](https://www.chromium.org/Home) or [Chrome](https://www.google.com/chrome)
- [Node.js](https://nodejs.org)
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
## Usage
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
There are three supported modes of running the tests.
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
#### Headless
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
The Selenium tests default to headless mode, unless a `DISPLAY` environment variable is set.
This variable may be set on Linux desktop and XQuartz environments. To run headless there,
unset the `DISPLAY` environment variable first.
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
npm run selenium-test
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
Or:
DISPLAY= npm run selenium-test
### Visible browser
To see the browser window, ensure the `DISPLAY` variable is set. On Linux desktop and in XQuartz
environments this is probably set already. On macOS, set it to a dummy value like `1`.
DISPLAY=1 npm run selenium-test
### Video recording
To capture a video, the tests have to run in the context of an X11 server, with the `DISPLAY`
environment variable set to its display name. If the shell has no X11 server or if you want
to hide the output, you can also launch a virtual X11 display using Xvfb. Recording videos
is currently supported only on Linux, and is triggered by the `DISPLAY` value starting with
a colon (as Xvfb typically would).
Example test run in [Fresh](https://gerrit.wikimedia.org/g/fresh).
fresh-node -env -net
export DISPLAY=:1
Xvfb "$DISPLAY" -screen 0 1280x1024x24 &
npm run selenium-test
## Filter
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
Run a specific spec:
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
npm run selenium-test -- --spec tests/selenium/specs/page.js
To filter by test case, e.g. with the name containing "preferences":
npm run selenium-test -- --mochaOpts.grep preferences
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
## Configuration
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
The following environment variables decide where to find MediaWiki and how to login:
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
- `MW_SERVER`: The value of `$wgServer`.
- `MW_SCRIPT_PATH`: The value of `$wgScriptPath`.
- `MEDIAWIKI_USER`: Username of a wiki account with sysop rights.
- `MEDIAWIKI_PASSWORD`: Password for this user.
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
## Further reading
Selenium tests in Node.js using WebdriverIO Introduce the WebdriverIO browser testing framework driven by Node.js. The overall intents are: * have MediaWiki core to provide a platform to run tests that is shared between core and the extensions. * phase out ruby driven browser tests eventually. Code is namespaced in sub directory /tests/selenium The 'pages' sub directory provides helper representing a MediaWiki page such as Special:Login and human friendly helpers to interact with the elements. Add Grunt task webdriver:test. Provide a npm script to easily spawn/dispose chromedriver and run above grunt task. wdio.conf.js provides all the configuration. It defaults to point to a MediaWiki-Vagrant installation on http://127.0.0.1:8080. Can be overriden with environment settings as needed. glob patterns (specs) are made absolute paths from MediaWiki root path that let us run the tests either from the root path (eg the npm wrapper or from the tests/selenium directory when invoking wdio directly. wdio assumes they are relative to the current working directory, hence the normalization. wdio.conf.jenkins.js extends the configuration and is used solely for Jenkins. The switch is done from the Gruntfile.js whenever JENKINS_HOME is set. Specially we want junit reports to be generated. Provide a more specific eslint configuration. References: * MALU https://phabricator.wikimedia.org/source/malu/ * T151442 Research WebdriverIO * T151443 Research Nightwatch.js Bug: T139740 Signed-off-by: Antoine Musso <hashar@free.fr> Change-Id: Ibe7a004a120e82af637ab3e31b725de743134c99
2016-12-19 16:39:29 +00:00
- [Selenium](https://www.mediawiki.org/wiki/Selenium) on mediawiki.org