Opened 4 years ago

Closed 3 years ago

#16009 closed task (fixed)

Move away from Mozmill to Marionette for Tor Browser tests

Reported by: gk Owned by: boklm
Priority: High Milestone:
Component: Applications/Quality Assurance and Testing Version:
Severity: Normal Keywords:
Cc: mikeperry, brade, mcs Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Mozilla is not maintaining Mozmill anymore, in fact deprecating it. They are instead using Marionette. We should port our tests to Marionette as well and migrate to that framework in order to avoid weird Mozmill bugs which are not getting fixed anymore.

See: https://github.com/mozilla/firefox-ui-tests for the new test suite and

https://firefox-puppeteer.readthedocs.org/en/latest/
https://marionette-client.readthedocs.org/en/latest/

for some documentation.

Child Tickets

Change History (13)

comment:2 Changed 4 years ago by mcs

Cc: brade mcs added

comment:3 Changed 4 years ago by boklm

I started adding support for marionette tests in our test suite, in the marionette_tests branch on my github repo:
https://github.com/boklm/tbb-testsuite/commits/marionette_tests

The tests can use the Firefox Puppeteer library:
https://firefox-puppeteer.readthedocs.org/en/latest/index.html

comment:4 Changed 4 years ago by gk

Priority: MediumHigh
Severity: Normal

Bumping priority given #17567.

comment:5 Changed 3 years ago by boklm

I pushed a marionette_tests-v3 branch on my github repo:
https://github.com/boklm/tbb-testsuite/commits/marionette_tests-v3

This adds a few fixes and convert all selenium tests to marionette.

The next steps will be:

  • adding a few Windows fixes, before merging the branch to master
  • adding a mechanism to pass some parameters to the tests (such as the Tor Browser version we are testing) similar to what we currently do for mozmill tests
  • converting all our mozmill tests
  • making it work on OSX

comment:6 Changed 3 years ago by boklm

I have now pushed some commits to the marionette_tests-v3 branch with some fixes to make it work on Windows.

comment:7 in reply to:  5 Changed 3 years ago by boklm

Replying to boklm:

  • adding a mechanism to pass some parameters to the tests (such as the Tor Browser version we are testing) similar to what we currently do for mozmill tests

This is now done in commit fabe206d932827663f17af099e4042623cd270d1 in the marionette_tests-v3 branch.

  • converting all our mozmill tests

The settings test is now converted in the marionette_tests-v3 branch.

comment:8 in reply to:  5 Changed 3 years ago by boklm

Replying to boklm:

  • converting all our mozmill tests

https-everywhere and play_videos tests have been converted to marionette.

  • making it work on OSX

Support for OSX has been added to the marionette_tests-v3 branch.

comment:9 Changed 3 years ago by boklm

I rebased the marionette_tests-v3 branch and merged it to master. The navigation-timing and resource-timing tests have also been converted to marionette.

comment:10 Changed 3 years ago by boklm

The acid3 and searchengines tests are now converted to marionette.

comment:11 Changed 3 years ago by boklm

The slider_settings and dom-objects-enumeration are now converted to marionette.

The following tests still need to be converted:

  • noscript (almost done)
  • screenshot
  • svg-disable
  • svg-enable

comment:12 in reply to:  11 Changed 3 years ago by boklm

Replying to boklm:

The slider_settings and dom-objects-enumeration are now converted to marionette.

After converting the dom-objects-enumeration test to marionette, we see a DOMConstructor object that we didn't see when using mozmill. I have added it to the list of expected objects to avoid an error.

comment:13 Changed 3 years ago by boklm

Resolution: fixed
Status: newclosed

All Mozmil tests have now been converted to Marionette.

Note: See TracTickets for help on using tickets.