Commit graph

128 commits

Author SHA1 Message Date
Bartosz Dziewoński
bde7249241 selenium: Remove '☃' character (U+2603 snowman emoji) from random usernames
The AntiSpoof extension disallows this character in usernames, causing
tests to fail on some setups. Its error message claims it is an
"unassigned character" (which is false), but it seems it's
intentionally disallowed and the error message should instead say
"non-script character" or something.

Bug: T198384
Change-Id: Ia8e7b425b6a3cf3742c00178f28c97a19f46bab0
2018-06-29 16:58:36 +02:00
Timo Tijhof
cee33128c8 wdio-mediawiki: Tag v0.2.0
Change-Id: Ia975b2d35220a6ad8ca90e6198ca2ba74c256ea3
2018-06-25 21:57:32 +01:00
Jakob Warkotsch
56b2412d6a Browser tests: use Util.getTestString in specialwatchlist.js
Follow-up on I5c5e9b22af5406f347636b22e68657b2674db6c9 and
I0747c2d361e951f8b3765a3dca548e55edb72216

Change-Id: I35ad793e6d9fe4c3bf58df9b4fcf6471c0a81f70
2018-06-25 12:14:25 +02:00
jenkins-bot
3aa8c6a332 Merge "Browser tests: extract commonly used getTestString function" 2018-06-25 09:51:52 +00:00
jenkins-bot
22ff706ce0 Merge "Add browser test for basic watchlist functionality" 2018-06-25 09:49:02 +00:00
jenkins-bot
1c944a2700 Merge "Selenium: run Wikibase repo tests" 2018-06-20 11:45:55 +00:00
Jakob Warkotsch
d96ca0d6a9 Browser tests: extract commonly used getTestString function
Extracting this function into a Util module reduces redundancy for
existing and future tests in core and extensions.

Change-Id: I5c5e9b22af5406f347636b22e68657b2674db6c9
2018-06-20 13:06:45 +02:00
Željko Filipin
6ebb3a5573 Selenium: run Wikibase repo tests
Similar to custom location of VisualEditor tests, Wikibase has tests in repo and client folders.

Bug: T180228
Change-Id: I3debe8bed9a760aea374529dbe007aa5a72421aa
2018-06-19 17:04:09 +02:00
Jakob Warkotsch
b25d303007 Add browser test for basic watchlist functionality
Will be used to test more advanced title/link formatters downstream.

Bug: T191601
Change-Id: I0747c2d361e951f8b3765a3dca548e55edb72216
2018-06-19 16:24:48 +02:00
Pablo Grass
97b6792142 selenium: RecentChanges page object
Return DOM element(s) to specs instead of creating another level of
inflection and to be more in line with other implementations.
Follow-up for I916516d5055c3ce9be071a75e5c3af2b3547d4ba

Bug: T191600
Change-Id: I03b0cd4af7adca23dd34173710b995629f833a1a
2018-06-19 15:31:07 +02:00
Pablo Grass
948990f05d selenium: add RecentChanges test to page spec
Add Special:RecentChanges page object with method to retrieve
displayed titles, test page last edited is shown there.
Will be used to test more advanced title/link formatters downstream.

Bug: T191600
Change-Id: I916516d5055c3ce9be071a75e5c3af2b3547d4ba
2018-06-13 16:23:20 +02:00
WMDE-Fisch
3ad98462e7 Use more diverse unicode chars in basic browser tests
See comment in I3423011c467b0a6426cfa0dad522435618f24bd0
and https://mathiasbynens.be/notes/javascript-unicode#poo-test

Decided on using something more positive than a pile of poo though.

Change-Id: Ic34d0d78044cede71fd1cd8742214c117ed2e99f
2018-06-13 14:59:46 +02:00
addshore
8602e489e1 selenium, page spec, make "should be editable" actually edit
This was using the same content for the edit before this
patch, so no edit was actually made, (just a null edit).

Change-Id: I945090647226e82f22b5b10a414a2a0bf9f2bc19
2018-06-13 11:56:29 +00:00
addshore
09536c527d selenium: page spec, make it easier to see where strings come from
Having randomly generated strings is great, but sometimes
it can help to allow a human to easily see where the string
is actually being generated in code, so add an optional suffix
to the string being generated.

Change-Id: Ibfe45f74b4880a70a76c01223a787dabcd43607e
2018-06-13 11:56:23 +00:00
addshore
ea4e00d590 Use wd/hub not /wd/hub in selenium.sh
I have run into this before and we fixed it in the README
but not in this script (I wasn't using this script at the time)

This was changed in the README in change:
I85a9e7dbd9a66418c85585969adb5ac1548f5ef6

Bug: T182421
Change-Id: I6ab92c6a61a57dc313b05321d0789dcbeb776c94
2018-06-13 11:56:15 +00:00
addshore
ae5f5b84c2 Don't use killall in selenium.sh
git bash for windows apparently doesn't have killall
by default, so instead get the PID of the chrome driver
we start and kill that specifically.

Change-Id: I8a9a773a7d3dc23807dc3ec12f5b3ef03661c9e3
2018-06-13 11:55:59 +00:00
Ed Sanders
4ffbfabf16 Selenium tests: Use assert.strictEqual instead of assert.equal
assert.equal is deprecated

Change-Id: I2f83d596db2ad475de5e005baa849782261d433e
2018-06-06 11:53:29 +01:00
Timo Tijhof
16a4d9da7b selenium: Initial version of wdio-mediawiki package
This is new package will be reusable by other repositories for
their browser tests, without having to reference the internal
selenium/pageobjects/ directory from MediaWiki core.

In addition to not requiring direct imports, it will also avoid
problems in the future by allowing the package to be versioned
and iterated upon without forcing an atomic global upgrade
(or broken master builds), everytime we change something.

See wdio-mediawiki/README for details.

Within MediaWiki core itself, the package is used using the
'file' specifier in its package.json, so that we always test
and develop using its working copy, which makes drafting and
testing changes easier.

Also misc changes to make wdio.conf easier to understand.

Bug: T193088
Change-Id: I547a7899e7a97693a93567dd763784e637433d55
2018-05-09 14:43:30 +00:00
Timo Tijhof
e52b056322 selenium: Create local ./log directory if needed
Without this, the tests sometimes fail like this:

> Error: ENOENT: no such file or directory, open './log/should-be-creatable.png'
>   at screenshot() - saveScreenshot.js:52:17
>   at saveScreenshot("./log/should-be-creatable.png")

This seems to race with the junit plugin, which uses mkdirp to
create it if missing, but the screenshot handling is separate
from that. WebdriverIO's own screenshot handling also does this
so it makes sense for ours to do that, too.

I considered trying to re-use WebdriverIO's save mechanism,
directly but it's not publicly exposed and only used for the
crash scenario, so for now we'll have to keep our own.

Also:
* Add to gitignore.

* Update default to use __dirname instead of './' because the
  latter will somtimes be mediawiki/ and sometimes be selenium/
  depending on whether you run all tests or some tests.

* Remove trailing slash from default logPath, and instead add the
  slash in filePath.
  Reason:
  - The LOG_DIR used by Jenkins doesn't end in a slash either
    (currently not failing because we no longer use that job,
    and let quibble run the tests instead, which doesn't set
    the LOG_DIR).
  - The WDIO docs and example also use screenshotPath without
    trailing slash.
  - Without this, setting LOG_DIR=/tmp/something results in
    filenames like /tmp/somethingexample.png.

Bug: T193088
Change-Id: I6550f9315bae89f96a791f7ae8cc2fbec5ee8dd5
2018-05-09 14:30:55 +00:00
Timo Tijhof
2729bf4653 selenium: Minor clean-up in preparation for packaging
This is functionally a no-op, purely refactoring (mostly style).

* Consistently require packages at the top of a file.
  (e.g. MWBot in edit.page.js).

* Remove unused .call(this) from mwbot interaction closures,
  which didn't use 'this'.

* Use Node.js regular Promise chaining with then(), instead of
  complex bluebird.coroutine generator function yields, which
  are intended to emulate async-await, but the syntax is quite
  error-prone for inexperienced developers and hard to debug.
  Once we require Node 7+ for the selenium tests, we can use
  async-await here natively, but until then, might as well use
  regular then() syntax, which we already use elsewhere in the
  tests, and is also what MWBot documentation uses.

* Also applied some minor whitespace changes for consistency
  among these files and other MediaWiki JS. E.g. no empty line
  before the first statement of a function. Add a new line between
  different methods, and between the end of a class and the
  export statement.

* Remove 'use strict' from test files. The patterns that would expose
  the bad non-strict behaviour are mostly already forbidden by ESLint,
  and the run-time optimisation to disable non-strict can't be noticed
  in tests (more useful in prod where e.g. the same process would run
  a function 1 million times). Main reason here is to keep things
  simple for new-comers and reduce boilerplate, given that these tests
  will mainly be worked on by browser-JS developers, not Node.js devs,
  and we don't currently use strict mode in our front-end code, either.

* Remove unused bluebird dependency.

Bug: T193088
Change-Id: I59f9211299e8e884c28c7733bcee3b7b28542610
2018-05-08 14:42:59 +00:00
jenkins-bot
02415cb826 Merge "selenium: Run wdio directly without grunt" 2018-05-03 07:29:40 +00:00
Timo Tijhof
495a96b51f selenium: Apply relPath() to the exclude option for CirrusSearch
It seems relPath() is used for all other paths here. When I was
running a single test from the /tests/selenium, I noticed wdio
was logging to the console that it excluded ./extensions/CirrusSearch,
which doesn't make sense from that directory. The other paths
are fine.

Change-Id: Ic89ae0a91cdb6590b7190ea44969e653e9e2d6ea
2018-05-02 16:50:50 +00:00
Timo Tijhof
9f85d84ec3 selenium: Increase mochaOpts.timeout from 20s to 60s
I was unable to get the tests passing locally in MediaWiki-Vagrant.
The main slowdown appears to be from MWBot logging-in for each
command anew. Which works fine, but just seems to be quite slow.

Change-Id: I3c0abc79cac48c26d952b0bbe73574aa48b2d70e
2018-05-02 14:50:19 +00:00
Timo Tijhof
810afd33e0 selenium: Run wdio directly without grunt
Doesn't seem to add anything, except complexity.

This has the benefit of matching more closely the way the README
recommends running individual tests.

Also add a check for 'chromedriver' before running it.
Normally the -e and pipefail would suffice, but because
it runs in the background, the error can be buried,
hard to find, or even not cause exit code to be set
properly. Thus, do a simple 'hash' check that will
print a useful command and exit cleanly.

Before:
> $ something &
> [1] 57922
> -bash: something: command not found
> [1]+  Exit 127                something
> $ echo $?
> 0

After:
> $ hash something
> -bash: hash: something: not found
> (exit: 1) $ echo $?
> 1

Change-Id: Id95f18927b5443defe679a77a82c5cbdd127c716
2018-04-30 21:54:15 +00:00
Timo Tijhof
ecd1f1bc90 selenium: Clean up wdio.conf.js
* Remove redundant user/pass for Jenkins, which is unused now
  that Jenkins jobs set it through ENV instead.
  See also T188520 and Ibb388741fee9 (integration/config).

* Use simple default operator (||) instead of elaborate
  undefined check.

* Document that the inline user/password defaults relate to
  MediaWiki vagrant.

* Set 'logPath' for Jenkins based on LOG_PATH env (from mw-set-env.sh),
  instead of assuming where it will be relative to mediawiki-core.

* Put new lines between blocks and functions.

* Document --no-sandbox hack from 4ff5652501.

Change-Id: I4c7856d260ded8c94e991a27863b80c9eea2e33e
2018-04-26 15:53:46 +00:00
Timo Tijhof
c0600fd29c selenium: Abstract test command behind 'npm run'
This makes it possible to use in Quibble. Right now, Quibble
cannot use 'npm run selenium' yet because it sets up its own
chromedriver, which would conflict with selenium.sh.

But, in preparation for Id95f18927b5, we can at least abstract
the test command so that it can easily be changed without having
to update Quibble and without having to worry about it being the
same in all MediaWiki branches.

Bug: T179190
Change-Id: I622ec3bf36746502cae891cc6bec23982b21f876
2018-04-23 15:56:54 +01:00
Željko Filipin
4cbd36db87 selenium: Remove Jenkins configuration file
Both Mocha and Cucumber tests should run. Since WebdriverIO supports only
running one test framework, the only way to do it is to have a generic
configuration file and two framework-specific files (Mocha, Cucumber). Having
Jenkins configuration file complicates things, and there is no need for
it.

Bug: T179190
Change-Id: I710066f7b5479dcad27aa57cd61007c1c2d88931
2018-04-13 20:54:43 +00:00
Željko Filipin
d972780d8b Selenium: move logic for running tests from package.json to selenium.sh
Just running `npm run selenium` in CI does not work. If WebdriverIO fails, the
job still passes since the last command to run is  `killall chromedriver`.
Reusing the script from CI that starts and stops Chromedriver should fix the
problem.

Bug: T179190
Change-Id: If29227263d23a3e4f26e2329dfa13c49f976cf8e
2018-04-13 14:21:29 +00:00
Antoine Musso
4ff5652501 Selenium: pass -no-sandbox to Chrome under Docker
When running webdriver.io in a Jessie Docker container, the sandboxing
fails:

  Failed to move to new namespace: PID namespaces supported, Network
  namespace supported, but failed: errno = Operation not permitted

Detect whether we are in a Docker container by checking /.dockerenv and
if present append --no-sandbox to Chrome options.

Change-Id: Ie5c276d12553e4473c01b3407e0f7622e0a678e0
2018-04-12 14:26:13 +02:00
jenkins-bot
9b00306abe Merge "WebdriverIO should be able to run Chrome headlessly" 2018-04-10 16:39:17 +00:00
Željko Filipin
36c0e76874 Selenium: use default value for waitforTimeout
The default is 10000 (10 seconds). It was increased by mistake. 20000 is the value needed for mochaOpts timeout. There is no reason to have the double value for waitforTimeout.

Bug: T179190
Change-Id: I14f72367a36815d155cedc57757481f6294fdc51
2018-04-06 13:08:20 +02:00
Željko Filipin
f195a217b4 Selenium: update WebdriverIO configuration
No changes are made to the configuration file. It has just been
regenerated with the latest WebdriverIO version. Minor whitespace
cleanup has been made to the file after it has been generated so it
conforms to our ESLint configuration.

This is the first step in updating configuration so videos of failed
tests are recorded.

Bug: T179188
Change-Id: Id0db3743ae7d828357b1527770bb7dcd2e02f99b
2018-03-30 17:29:47 +02:00
Željko Filipin
e60074cecf WebdriverIO should be able to run Chrome headlessly
Chrome 59 introduced an headless mode, in this mode it does not need any
DISPLAY (Xorg, Xvfb...). That achieved by passing to chrome:

 --headless

Use headless when DISPLAY is not available. One can switch to headless
by simply unsetting the environment variable: unset(DISPLAY).

Bug: T167507
Signed-off-by: Antoine Musso <hashar@free.fr>
Change-Id: Id5424eafb20112fd74d472b8cef02cb15cafff89
2018-03-29 15:18:00 +00:00
jenkins-bot
33fa8c095c Merge "Selenium: Replace nodemw with mwbot" 2018-03-21 15:17:06 +00:00
jenkins-bot
7a7accf2f3 Merge "Selenium: Do not run CirrusSearch smoke tests in Jenkins" 2018-03-21 14:42:37 +00:00
Željko Filipin
9c5fa5567b Selenium: Replace nodemw with mwbot
Bug: T181284
Change-Id: I012a5cc6a7f30a8652cbad87293eb119fa2e405f
Depends-On: Ic69be92c2b0b7a08a9b8cc417a40ebb38f249ea3
2018-03-21 13:34:37 +01:00
Željko Filipin
771c5a7814 Selenium: Upgrade WebdriverIO to 4.12.0
Bug: T180144
Change-Id: I8d34e2240fe76798c99315a54ea5bab63b2409e9
Depends-On: Ic69be92c2b0b7a08a9b8cc417a40ebb38f249ea3
2018-03-21 12:20:42 +00:00
Željko Filipin
d197371a0a Selenium: Do not run CirrusSearch smoke tests in Jenkins
Bug: T190039
Change-Id: I3b6ce071ce69ef3d6ea719d875428422f9ad7e6d
Depends-On: Ic69be92c2b0b7a08a9b8cc417a40ebb38f249ea3
2018-03-21 12:18:00 +00:00
Zfilipin
b6c4f60b0a Revert "Selenium: configuration file for beta cluster"
This reverts commit 3d2395a2af.

Change-Id: I4ad8958df097a1ef011c0f9d99ea9f7a0030c032
2018-03-20 11:12:27 +00:00
Željko Filipin
3d2395a2af Selenium: configuration file for beta cluster
Bug: T185011
Change-Id: Icf8588f0e82acf09e6654dd5dbf33479e4ecbbb7
2018-03-16 18:30:38 +01:00
Željko Filipin
30ebe529c0 Update README file for Selenium tests
Bug: T187862
Change-Id: Iadd41307a48a1b50f64a495e7efcee3b88d0ff68
2018-02-21 12:28:49 +01:00
Jdlrobson
33ba3cfebb Revert "selenium: add new message banner test to user spec"
This reverts commit 6ca3e221d1.
This is breaking CI for the Minerva skin and Echo as Echo changes how
this interaction works. When the browser test tries to locate the
div.usermessage it has moved so fails.

This test needs to be rewritten to take this into account.

On a side note, given usermessage is generated by all skins (and not
in core) there is a big argument to be made that this shouldn't be in
core but in the individual skins. MinervaNeue the mobile skin notably
does not use this class so this test would always fail on a mediawiki
instance which only uses the Minerva skin.

Bug: T185928
Change-Id: I43b6cf328aaf4a4fded0bb9746716c4cc9d8f364
2018-01-29 20:41:21 +00:00
addshore
6ca3e221d1 selenium: add new message banner test to user spec
Change-Id: I92dc612d215d1d3eb8dff02c2ff4d57da0105d69
2018-01-24 14:06:01 -08:00
addshore
817baf7e91 selenium: add re-creatable page test to page spec
Change-Id: I61c452246c29bd22000a06844901dfe8ca06fadd
2018-01-20 10:47:59 +00:00
addshore
9c7d876341 selenium, add restoration test to page spec
Change-Id: Ied06915f982e693b00edab14b8c29c978a5d6dff
2018-01-20 10:45:16 +00:00
addshore
52c7dc56df selenium: add deletion test to page spec
Change-Id: Ia17b52958778a09378eeb10c9f24ebe9229bd9f5
2018-01-20 10:41:41 +00:00
Željko Filipin
6e1b37c966 Page title is not used in Selenium tests
Bug: T182421
Change-Id: I0de9bf27f1227e7b29c5aaa28451823e80fa8ff1
2018-01-17 13:15:56 +01:00
Željko Filipin
ee6a408527 When running Selenium tests, use wd/hub
It works both on Mac and Windows. I am assuming it works on Linux too.

Bug: T182421
Change-Id: I85a9e7dbd9a66418c85585969adb5ac1548f5ef6
2018-01-16 18:40:33 +01:00
addshore
877c2e0a58 selenium, update page spec to include more chars
Bug: T184749
Change-Id: I3423011c467b0a6426cfa0dad522435618f24bd0
2018-01-15 10:36:34 +00:00
addshore
cff382d605 Selenium README note about --url-base on Windows
For me /wd/hub doesnt work and wd/hub must be used

Change-Id: Ic61d516b53c95b0d1351bcda7308870f5fc5171c
2018-01-15 10:03:00 +00:00