Closed Bug 1066573 Opened 5 years ago Closed 5 years ago

Failures in mozmill tests due to e10s on nightly

Categories

(Mozilla QA Graveyard :: Mozmill Tests, defect)

defect
Not set

Tracking

(firefox35 affected)

RESOLVED FIXED
Tracking Status
firefox35 --- affected

People

(Reporter: AndreeaMatei, Assigned: whimboo)

References

(Blocks 1 open bug)

Details

(Keywords: regression, Whiteboard: [mozmill-test-failure])

e10s being included in Nightly will break some of our tests, until we make use of Mozmill 2.0.7. 
Keeping track here of the failing tests, so far:
Notification popup state has been opened - restartTests/testAddons_InstallAddonWithoutEULA/test1.js
http://mozmill-daily.blargon7.com/#/remote/report/b509c8df3c5f32f8b1b39e28bee35173
Sadly we have only been informed a couple of hours before this change went life in Nightly. There was no way to react that quickly, to not see lots of our tests failing due to the changes in bug 1064885. :( I really would appreciate to get a heads-up information 2 or more days before. Actually this is again a downside not being on TBPL.

So as for the status update, mozmill 2.0.7 got released. We are working on the mozmill-automation 2.0.7 release (https://github.com/mozilla/mozmill-automation/issues/170) now. Once this is done new environments will be created and mozmill-ci staging be updated. I don't want to upgrade production, given that QA has to run beta update tests today. So not earlier than Monday we will be able to run Mozmill tests successfully against Nightly.
Blocks: 1064885
Keywords: regression
Sorry again, Henrik. I was not familiar with how Mozmill works.
Blocks: e10s-tests
(In reply to Chris Peterson (:cpeterson) from comment #2)
> Sorry again, Henrik. I was not familiar with how Mozmill works.

I understand. Looks like we simply faced a bad combination of situations. With also having our QA lead of this feature out last week, we simply got this info a bit too late through other channels. I would say at least we got it! So thanks to Gavin and Marc, who both pointed that out to me.


Ok, so after kinda hard work on Friday, I haven't seen any failures for recent Nightly builds anymore. So I pushed the upgrade to version 2.0.7 to our production instance. As seen right now, all tests are passing again, and none is showing the issues as we have seen here for e10s.

Lets try to make Mozmill ready for e10s at latest early October.

Other than that everything else on this bug is done.
Status: NEW → RESOLVED
Closed: 5 years ago
Depends on: 980801
Resolution: --- → FIXED
Assignee: nobody → hskupin
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.