1165215 - Update all jobs to flip from Mozmill tests to Marionette tests

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Infrastructure, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We want to replace all Mozmill jobs with the appropriate Marionette ones. This will happen across branches except for mozilla-esr31. On that latter branch Marionette is not supported. It will continue to run Mozmill until its end of life.

Things to do:
* Prepare all machines for system level changes (see bug 1163942)
* Update all jobs in mozmill-ci for Marionette (see https://github.com/mozilla/mozmill-ci/issues/576)

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

While working on this bug I'm going to discontinue the endurance and addon tests, and remove them from the Mozmill CI config. A patch for all jobs for Nightly builds will be up in a bit on the github repo.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Together with Barbara we figured out some remaining issues on the firefox-ui-tests master branch. Both got fixed, so we are good to get Marionette tests running in production now for Firefox Nightly builds. I'm going to push it in a bit.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Tests for Aurora look pretty good. It seems there is only bug 1168880 which needs to be fixed to get our tests passing on localized builds. There might be one more for test_places.py which needs to be disabled for XVFB on Travis. I will check that next.

I'm targeting to enable Marionette tests for mozilla-aurora on staging by latest tomorrow.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We still have some bugs to get fixed before we can enable Aurora in staging. First there is a major regression in Marionette (bug 1169798) to be fixed down to Beta. Otherwise also lots of jobs are failing due to networking issues when creating the environment. PyPI is not trustworthy so we might also have to flip to the internal pypi mirror from Releng. For more details see https://github.com/mozilla/mozmill-ci/issues/586.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

All deps have been solved and beside esr31 and ondemand all jobs are running firefox-ui-tests now. All in all the results are very stable and we get nearly no failures.

I call this done!