Commit graph

72 commits

Author SHA1 Message Date
DannyS712
676d8a6488 Selenium: provide CreateAccountPage in wdio-mediawiki
So that it can be used by extensions too. Use case:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/702893

Change-Id: Ie1c71484526718b91c43ae881b0230ac4f1cef61
2021-07-08 03:50:24 +00:00
DannyS712
4105ad3ef6 Selenium watchlist tests: don't create a new account
Use existing admin account

Change-Id: I316de111341c37a462c883fccdad9db634204fc5
2021-06-09 12:05:27 +00:00
Petr Pchelko
ff4e6f59dc Add selenium tests for page protection
Change-Id: I31349f747dc1fa22c3bfbb3fe02386686aeb2cf1
2021-06-07 11:35:02 -07:00
Željko Filipin
00582fa7a8 selenium: Simplify spec file names
Remove `special` from `recentchanges.js` and `watchlist.js`.
The entire `spec` folder has only four files. The two mentioned and
`page.js` and `user.js`. If that was a folder with a lot of files, it
would make sense to use `special`. Now it just makes file names long.
Additionally, files in `pageobjects` folder are already using simpler
names, `recentchanges.page.js` and `watchlist.page.js`, without
`special`.

Bug: T210726
Change-Id: I156f436e2de345586a25174c630ad60259504115
2021-05-20 19:46:09 +02:00
Željko Filipin
24e2f591f2 selenium: Delete tests disabled for more than 1 year
Rollback with confirmation
- should offer rollback options for admin users (Kosta Harlan, 2 years ago, 2fb62de)
- should offer a way to cancel rollbacks (Adam Roses Wight, 2 years ago, 89a00d2)
- should perform rollbacks after confirming intention (Jon Robson, 2 years ago, ddf7a50)
- should verify rollbacks via GET requests are confirmed on a follow-up page (Timo Tijhof, 2 years ago, c3e379d)

Rollback without confirmation
- should perform rollback via POST request without asking the user to confirm (Sam Reed, 2 years ago, 8631daa)
- should perform rollback via GET request without asking the user to confirm (Kosta Harlan, 2 years ago, f0162af)

User
- should be able to change preferences (Kunal Mehta, 3 years ago, 4b0cde2)

Bug: T280652
Change-Id: Ieaccf37ad016f2e811aeb98d2bd2f061c9b13fbf
2021-04-23 13:25:59 +02:00
Željko Filipin
0d3c17754c selenium: Fix browser.getText is not a function
Tests in rollback.js are disabled due to flakiness. After enabling them
a few of them fail with `browser.getText is not a function`.
It is caused by WebdriverIO API change. `browser.getText()` was replaced
with `$().getText()`. The problem was not noticed during the WebdriverIO
upgrade since the tests were disabled.

Bug: T280639
Change-Id: I4b21bbc1ad9ddbc8b04b444b21ee67b2621f54e7
2021-04-20 10:37:43 +00:00
Kosta Harlan
3244a9e367 selenium: Disable flaky undoable page test
Bug: T276783
Change-Id: Ic3f9dbd38511ffb503ffd341f8af4a32405ad1ed
2021-03-11 11:55:07 +01:00
Željko Filipin
5015a623d6 selenium: Remove ChromeDriver dependency
Since WebdriverIO v5, Puppeteer is available as a Chromedriver
alternative.

Puppeteer and Chromedriver behave almost the same, but there were a few
minor failures that needed to be fixed.

Puppeteer is bundled with WebdriverIO. Chromedriver needs to be installed
(and started/stopped) separately. Getting rid of Chromedriver simplifies
our documentation, among other things.

The commit updates tests/selenium/wdio.conf.js to use Puppeteer.
It also fixes a few minor problems in tests/selenium/specs/page.js
introduced by the change.

Bug: T269566
Change-Id: I143c44304461980a2be98eb60741184fa7671343
2021-03-07 02:31:06 +00:00
Adam Wight
bc4d631c08 Wait until the recent changes are updated
There may be a race condition when running against a multithreaded
server, in which the Special:RecentChanges request arrives before the
deferred job has updated history.  The API call ensures that the
recentchanges table has been updated, within a 5-second grace period.

Bug: T225218
Change-Id: I0882a5624c62b94f8c6e6056ea3396d9e2904d48
2021-01-05 16:49:27 +01:00
Ed Sanders
8b720e9bd6 eslint: Update to eslint-config-wikimedia 0.16.0
* valid-jsdoc replaced with jsdoc plugin
* New /selenium config

Change-Id: I471eebac0312cb25c539c3f6a3ecfc7cfd4ed8d6
2020-06-02 21:32:56 +01:00
Ed Sanders
d94f90e088 eslint: Code style fixes for upcoming config change
Change-Id: Ief44c0bb26c5a1c1773cab5dba51d5e856612bf7
2020-05-14 22:19:55 +01:00
Kosta Harlan
0257160852 [selenium] Make tests skin agnostic
The current tests in core rely on selectors provided by Vector, but with some
small modifications the tests can work equally well with and without Vector.

Bug: T248484
Change-Id: I5bc0c80b796252b1a920f6549ef4a462a9b13b87
2020-03-25 16:21:18 +01:00
Reedy
8631daae64 Disable flaky Selenium test
Bug: T226144
Change-Id: Ib49898617ba419f709b249ae9a05d9cb7b8e4e9d
2019-10-08 20:50:37 +00:00
Timo Tijhof
22d149c3f7 selenium: Fix more inefficient MWBot use and simplify wdio-mediawiki Api
This does the same for the other specs, as previously done to the
page.js spec in 058d5b7cd8.

* rollkback: From 6 api logins to 4 api logins.
  Before (2x3): admin for edit, admin for createaccount, vandal for edit.
  After (2x2): admin for edit + createaccount, vandal for edit.

* recentchanges spec: No difference, but updated pattern for consistency
  so that if it is extended in the future, it will be natural to re-use
  the bot object instead of creating a new one.

* watchlist spec: From 3 api logins to 1 api login.
  Before: admin for createaccount, admin for edit, admin for edit.
  After: admin (re-used)

* user spec: From 2 to 1 api login.

Also:

* Remove the now-unused Api.edit() and Api.delete() anti-pattern
  methods, as these are nothing but one-line shortcuts to the
  already one-line invocation of bot.edit() and bot.delete(),
  except that they bypassed the current bot object, causing
  inefficient repeat logins in way that was non-obvious.
  Migration is simple and won't be required until other repos
  upgrade to the next wdio-mediawiki version (not yet released).

* Make 'bot' a mandatory parameter for the createAccount, block,
  and unblock convenience wrapper methods.

* Move the vandalizePage() method from HistoryPage to rollback spec,
  as it had no connection with that page object or the action=history
  interface, and document why it can't (yet) re-use its bot object.

Bug: T234002
Change-Id: Id6e995916566f7dd7b618892295198b897fbee2e
2019-10-04 18:16:28 +01:00
Timo Tijhof
46586a243c selenium: Replace ES5 one-var assignments with const/let per line
The one-var rule only applies to 'var' where it matters that we
don't let the engine implicitly hoist variables leading to
confusing code that may be unsafe to refactor in a straight-forward
manner.

This doesn't apply to ES6 const and let. While I updated the
rule in 1955a8aa5 to allow ES6, I did not enforce it. Fixed now.
When assigning a value, each assignment should be its own
statement (disallow awkward comma separated assignments).
When not assigning a value, and the lines are next to each other,
they should be in one statement. This makes sense for this small
directory and might make sense to propose for the wikimedia preset
at some point.

Note that the rule I added was not required to make the build
pass. The changes I made are allowed by default by the wikimedia
preset for es6+. The rule I added disallows the previous way.

Change-Id: I26cda095a8a4589084e36273038877f2dd1fc50b
2019-10-01 02:18:19 +00:00
Kosta Harlan
640b0b4943 Selenium: Hide VE welcome and beta dialogs
Remove uses of setLocalStorage (incompatible with chromedriver >= 75)
and also requires slightly fewer requests since BlankPage.open()
doesn't need to be invoked.

Bug: T234002
Change-Id: I4cfb4fe17a3c99bfcb7ff39ac6db60f5c5e8a687
2019-09-30 21:57:23 +02:00
Timo Tijhof
058d5b7cd8 selenium: Fix inefficient use of MWBot in specs/page.js
This test previously took 4 minutes to run locally and ended up
running the MWBot.login process 9 times.

After this, the specs/page tests only log-in once.

Bug: T234002
Change-Id: I374620a01f49d4da559070d0982bdbe4c1269e2e
2019-09-30 14:47:00 +01:00
Timo Tijhof
4bd69c7536 selenium: Remove "RunJobs" wait from specialrecentchanges test
This test was consistently reaching the RunJobs timeout of
10 tries and then giving up. This happens in CI, where it wasted
about ~ 5 seconds, and locally where it takes about 30s for mw.

Like so:
> RunJobs through requests to the main page (run 1).
> RunJobs detected 12 more queued job(s).
> RunJobs through requests to the main page (run 2).
> RunJobs detected 12 more queued job(s).

Etc, until it reaches run 10 and gives up. This also suggests
the whole thing isn't wokring, but that's a separate matter.

First and foremost, RecentChange objects are saved from a PRESEND
deferred update, which means it does not require a job queue wait
in order to appear on Special:RecentChanges.

This was added in e908d347ef (T199446) "to keep the number of
unrun jobs low". The idea being that this test makes an edit via
the API, which means it creates jobs without running them. On
the other hand, there's also lots of test steps that view pages
but don't create jobs, which would make this optimisation premature.
If and when we do encounter a test failing due to a job backlog,
I'm confident we can find a better solution at that time.

Bug: T234002
Bug: T225730
Bug: T199446
Change-Id: I9ef8c5b65d4b636572569912f04f48ed40aa0564
2019-09-29 02:32:24 +00:00
Timo Tijhof
1955a8aa56 selenium: Upgrade from webdriver v4 to v5
* Options no longer needed or no longer exist in wdio v5:
  - coloredLogs: Now always on. The underlying 'chalk' library can still
    be influenced via the FORCE_COLOR environment variable.
  - screenshotPath: Removed. Was already disabled in our config.
  - deprecationWarnings: Meh.
  - 'sync: true' – On by default when `@wdio/sync` is installed.
    The wdio v5 config generator doesn't recommend setting manually.

* The selenium.sh script was removed. It existed to start and stop
  chromedriver for local use by developers. This is now done by
  the wdio-chromedriver-service. In WMF CI, Quibble starts its own
  chromedriver (as optimisation, reused across gated repos),
  which is why the 'selenium-test' entry points remains and skips
  this.

* The wdio-mediawiki package now requires wdio v5 and Node 10.
  This doesn't affect extension repos because versions are pinned.
  Upgrade may happen at the earliest convenience.

* Several WDIO methods changed names or signature. Full list at:
  <https://github.com/webdriverio/webdriverio/blob/v5.13.2/CHANGELOG.md#v500-2018-12-20>
  Highlights:
  - browser.element() is now browser.findElement() with "$()" as alias.
  - browser.localStorage replaced by browser.setLocalStorage.
  - browser.deleteCookie() requires `name` param. To delete all at once,
    there is a new method browser.deleteAllCookies().
  - Commands that return data no longer wrapped in `{ value: … }`.
    Values are now returned directly.
  - Custom config keys are now under browser.config instead of browser.options.
    Renamed our username/password keys to be mw-prefixed, to avoid clashes
    and reduce confusion with similar config keys.
  - browser.click(selector) and browser.getText(selector) no longer exist.
    Use $(selector).click() or .getText() instead.

* Fix "no such alert" warning from specs/page.js by removing the apparently
  redundant code.

Bug: T234002
Bug: T213268
Change-Id: I908997569ca8457997af30cb29e98ac41fae3b64
2019-09-29 02:59:44 +01:00
jdlrobson
aaea6a0fc0 Do not hardcode square brackets in "rollback link"
Additional change:
* standardize on "optional" lowercase in qqq messages

Bug: T219948
Change-Id: I7d79c0ca534bff7165975943d5fa22dfe0a616db
2019-07-18 14:30:23 -07:00
jenkins-bot
dfec83932f Merge "Selenium: replace UserLoginPage with BlankPage where possible" 2019-07-08 17:00:13 +00:00
Željko Filipin
97c13ffeb8 Selenium: replace UserLoginPage with BlankPage where possible
When using local storage, any page has to be open. UserLoginPage is usually used.
BlankPage should be slightly faster to load. It also makes it more clear than
any page would do the job.

Change-Id: I50b23993065ad6b093a6cc951d00b0bf7a3d5e68
2019-07-08 13:44:32 +00:00
Ed Sanders
689d07b81d build: Update eslint-config-wikimedia to 0.13.0
Change-Id: I7df5840bc97f8bbcd8cfc875c2b48b3061ef2c9e
2019-07-05 13:32:26 -07:00
Kosta Harlan
2fb62de70a selenium: Skip flaky rollback test
Bug: T219440
Change-Id: I2199b7e7d4f7ba98a80ddfc568cf9957eba36182
2019-06-27 17:21:08 +00:00
Kosta Harlan
f0162afbc9 Disable flaky Selenium test
Change-Id: Ia3f2403e452f4476238b349c9d6a6f8135267456
Follows-Up: Iaf7e095b3bb34072eea6bcac76ba29358b14cc09
Bug: T226144
2019-06-19 21:43:47 -04:00
Timo Tijhof
c3e379d603 selenium: Skip another flaky Rollback test
Bug: T220479
Change-Id: Ie523e719f5c7df959ef63930edaff1a80e09ff04
2019-06-13 18:22:11 +01:00
Adam Wight
89a00d2838 selenium: Skip flaky Rollback test
Sometimes the element doesn't appear.  This is breaking during gate-and-submit,
so skipping until it can be fixed properly.

Bug: T219440
Change-Id: I7286eb5499e34146acf5da3180b665436569a18c
2019-06-10 15:18:12 +00:00
jenkins-bot
35afb25421 Merge "Temporarily skip rollback test" 2019-05-07 21:50:52 +00:00
jdlrobson
ddf7a50057 Temporarily skip rollback test
This test is proving flakey (seen in Minerva and MobileFrontend)
Let's skip this until we've worked out the problem.

Bug: T220479
Change-Id: I3629d4109d8e0c5c4da0c7fb0b82cf59290c0db9
2019-05-07 13:47:58 -07:00
tzhelyazkova
3990e0bbc7 Improve rollback tests setup by extracting repeating logic to HistoryPage object
This also removes the helper method waitForScriptsToBeReady() and uses the more
general waitForModuleState() method directly.

Bug: T220479
Change-Id: I6be81ad68720449ef03b6a60b4b329040f236c72
2019-04-30 18:17:00 +02:00
jenkins-bot
a4777729dd Merge "Replace pauses in rollback tests w/ dynamic waits" 2019-04-30 14:03:50 +00:00
Tim Eulitz
52f095a844 Replace pauses in rollback tests w/ dynamic waits
Bug: T220479
Change-Id: I95cf06a4d6a677ca14b56f11f5c6bd98aa0abd05
2019-04-30 10:55:24 +02:00
jenkins-bot
19fe64a134 Merge "selenium: Replace regex matching with includes()" 2019-04-15 17:30:47 +00:00
Cormac Parle
a45f0e4613 selenium: Replace regex matching with includes()
Bug: T217544
Change-Id: Iedcbb1f7e21fc768069346ac346c2e499a689556
2019-04-15 17:09:12 +00:00
Timo Tijhof
ec4be6b44f selenium: Disable flaky wdio rollback tests that rely on pauses
Wait for an actual state to be reached, whether a JavaScript
expression to become true, or a certain state in the DOM.

Using pauses is an anti-pattern and inherently unstable.

Bug: T219440
Change-Id: Ibb2bd335f12442ab8ec2973031454cb58733211d
2019-04-14 23:12:53 +01:00
Cormac Parle
d653232b22 Selenium: fix 'Page should be editable' test
The test is broken only when and targeting beta cluster, because an
extension adds text to page. Instead of checking the entire body of
the page, test just checks if expected string is there.

Bug: T217544
Change-Id: Ibd67790c1df2cd8556b9c71d872d52cfc18f5ebd
2019-04-11 16:37:58 +00:00
Tim Eulitz
684e544f1a selenium: Add API wait and other pauses in rollback test
Bug: T219440
Change-Id: Ifdc0ff72db7f6f176b8a7a0522235e91dfd2aa87
2019-04-09 10:18:26 +00:00
tzhelyazkova
e3593feb05 Fix flaky rollback browser test
Bug: T219440
Change-Id: I58c21494201d4fea738a9e9e007faf272e89c6e5
2019-03-29 15:32:02 +01:00
jdlrobson
063cce733e History page is now using pseudo elements for presentation
|, ( and ) separators are now added via CSS rather than being
made part of the HTML.

Bug: T205581
Change-Id: I272770418ee544b22f8f94312daf07010b7a2273
2019-03-26 00:15:21 +00:00
Tim Eulitz
a2569497f6 Use delegated events for rollback confirmable
This change is necessary to make the rollback confirmation work even
when the list of items which can be rolled back is updated via
JavaScript (this is possible on RecentChanges, for example).

Bug: T218354
Change-Id: Idafe1380b283cf0056ff1efdffa04aca92cf1217
2019-03-21 18:14:07 +01:00
Tim Eulitz
0a84032711 Implement non-JS RollbackAction with form
This change prepares a form for the RollbackAction to allow rollbacks to
be triggered via POST while also ensuring users are always prompted with
a request to confirm the rollback if the rollback confirmation prompt is
enabled.

Bug: T215303

Change-Id: Iaf7e095b3bb34072eea6bcac76ba29358b14cc09
2019-03-21 12:16:44 +01:00
Tim Eulitz
341320457c Show confirmation prompt on rollback links
Bug: T215020
Change-Id: Ic831888e30808a20a04397912498fe2ca04f80ba
2019-03-21 10:13:22 +00:00
Thiemo Kreuz
3b03ea9bdc Add browser test for preview functionality to MediaWiki core
The basic functionality of being able to preview an edit is currently
not covered by a test, as far as I can see.

The assertion for a wpTextbox2 that should *not* be there is a result of
the issue documented at T209012, where the EditPage::isConflict flag was
accidentially set. This assertion makes sure accidential conflicts can't
happen again, no matter which extension might cause it.

Bug: T210758
Change-Id: Iae723430b3a88079ad3499429e65c29817eca67e
2018-12-07 16:27:45 +01:00
Pablo Grass
e908d347ef selenium: invoke jobs to enforce eventual consistency
Many tests use the API during their setup phase, which does not
participate in clearing the job queue, but constantly adds new items.
Consequently the job count keeps mounting. In order to be able to assert
the state of the application once all jobs are processed we need a way
to invoke them.
This is a minimalistic alternative to Idd4924e16925cbf0cb9e42645670c3fd76b72353

Bug: T199446
Change-Id: Ifc49bc5c06eef71da8aff1b1837c32debc2c45dd
2018-09-20 11:13:38 +02:00
Željko Filipin
e552e95f71 Selenium: Daily Jenkins job targeting beta cluster should run only tests that pass
Selenium tests run fine when targeting MediaWiki installation in
MediaWiki-Vagrant and in Jenkins, but fail when targeting beta cluster.
Until tests are refactored to run without failure for beta cluster, let's disable
failing tests.

Bug: T185011
Change-Id: I12da893e7d624d4ebe478bccf0706aa07c965796
2018-09-04 12:47:07 +02:00
Pablo Grass
32c36f0e73 selenium: undo page
Add basic undo page object and test showing a previous edit can be
undone.

Bug: T201901
Change-Id: Idc1b3889abc6e655032bc1576f1b9d591abd6c8d
2018-08-14 16:27:31 +02:00
Kunal Mehta
4b0cde2a09 Disable "...able to change preferences" browser test
It's incredibly flaky.

Bug: T199446
Change-Id: I5372a285dab4f5f032ae340d9fd30b9c7c8bf72a
2018-07-31 12:28:31 -07:00
Daimona Eaytoy
6a17d6b3a1 Temporarily skip broken Selenium test
This test was already flaky and was fixed in
I1fd0c49263e975c6b24e2e1f9a072486a6565dda. However, this caused failures
on several repos. Temporarily disable it until we'll get it to work
properly.

Bug: T199644
Change-Id: I88be83e01eb1789f28f09fc4a0be7ea17f402858
2018-07-15 15:42:05 +02:00
Pablo Grass
42f844e04a selenium: create edit for RecentChanges through GUI
Use page object to create the page we will be asserting on
Special:RecentChanges. Apparently (post send) jobs are not executed on
API calls (that were formerly used for test setup) and consequently
presence of the edit on Special:RecentChanges could not be guaranteed.

Bug: T199446
Change-Id: I1fd0c49263e975c6b24e2e1f9a072486a6565dda
2018-07-13 17:21:25 +02:00
Stephane Bisson
aea721b883 Let 'shows page creation' test be skipped
Troubleshooting can happen in parallel without blocking
unrelated work from being merged.

Bug: T198137
Change-Id: Ie84a868a580968ff99ff6879050080c6495fa72a
2018-07-12 18:52:57 -07:00