Failures in mozmill tests due to e10s on nightly

RESOLVED FIXED

Status

Mozilla QA
Mozmill Tests
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: AndreeaMatei, Assigned: whimboo)

Tracking

(Blocks: 1 bug, {regression})

unspecified
regression
Dependency tree / graph

Firefox Tracking Flags

(firefox35 affected)

Details

(Whiteboard: [mozmill-test-failure])

(Reporter)

Description

3 years ago
e10s being included in Nightly will break some of our tests, until we make use of Mozmill 2.0.7. 
Keeping track here of the failing tests, so far:
Notification popup state has been opened - restartTests/testAddons_InstallAddonWithoutEULA/test1.js
http://mozmill-daily.blargon7.com/#/remote/report/b509c8df3c5f32f8b1b39e28bee35173
(Assignee)

Comment 1

3 years ago
Sadly we have only been informed a couple of hours before this change went life in Nightly. There was no way to react that quickly, to not see lots of our tests failing due to the changes in bug 1064885. :( I really would appreciate to get a heads-up information 2 or more days before. Actually this is again a downside not being on TBPL.

So as for the status update, mozmill 2.0.7 got released. We are working on the mozmill-automation 2.0.7 release (https://github.com/mozilla/mozmill-automation/issues/170) now. Once this is done new environments will be created and mozmill-ci staging be updated. I don't want to upgrade production, given that QA has to run beta update tests today. So not earlier than Monday we will be able to run Mozmill tests successfully against Nightly.
Blocks: 1064885
Keywords: regression
Sorry again, Henrik. I was not familiar with how Mozmill works.
Blocks: 984139
(Assignee)

Comment 3

3 years ago
(In reply to Chris Peterson (:cpeterson) from comment #2)
> Sorry again, Henrik. I was not familiar with how Mozmill works.

I understand. Looks like we simply faced a bad combination of situations. With also having our QA lead of this feature out last week, we simply got this info a bit too late through other channels. I would say at least we got it! So thanks to Gavin and Marc, who both pointed that out to me.


Ok, so after kinda hard work on Friday, I haven't seen any failures for recent Nightly builds anymore. So I pushed the upgrade to version 2.0.7 to our production instance. As seen right now, all tests are passing again, and none is showing the issues as we have seen here for e10s.

Lets try to make Mozmill ready for e10s at latest early October.

Other than that everything else on this bug is done.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Depends on: 980801
Resolution: --- → FIXED
(Assignee)

Updated

3 years ago
Assignee: nobody → hskupin
You need to log in before you can comment on or make changes to this bug.