Test failure "Application crashed" in /restartTests/testAddons_uninstallExtension/test3.js

RESOLVED INVALID

Status

RESOLVED INVALID
4 years ago
a year ago

People

(Reporter: tracy, Unassigned)

Tracking

Version 2
x86
Mac OS X

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
Seeing this one frequently in Mozmill release-mozilla-beta_functional testrun for Firefox 33.0
Usually Daniel should have gone through all those reports today, and filed bugs appropriately. But sadly I can see your confusion here. There is not a single reply on the CI mailing list in case of crashes happened! This is bad! I mentioned this a couple of times, especially that this can cause confusion for QA people. Now we have such a situation. Please really try to get this solved for the duty.

Tracy, do you have some example jenkins urls? That would be very helpful! As of now it's hard to do action on it. So what has to be done here, is to login to the machine(s) and submit the crash reports. An upcoming version of Mozmill will be able to show the stack directly.
Flags: needinfo?(twalker)
(Reporter)

Comment 2

4 years ago
Here is the report for one such crash: http://mozmill-release.blargon7.com/#/functional/report/3dac09185ffafe61214f5a99be184340
Flags: needinfo?(twalker)

Comment 3

4 years ago
(In reply to Henrik Skupin (:whimboo) from comment #1)
> Usually Daniel should have gone through all those reports today, and filed
> bugs appropriately. But sadly I can see your confusion here. There is not a
> single reply on the CI mailing list in case of crashes happened! This is
> bad! I mentioned this a couple of times, especially that this can cause
> confusion for QA people. Now we have such a situation. Please really try to
> get this solved for the duty.

With the recent changes to run all (most) locales against beta, the CI machines are on full capacity most of the day. I remember this was discussed on IRC yesterday a bit. 

To properly investigate (and submit) a crash report we must take the affected machine offline, which we can not currently do without disrupting the testrun batch. With > 1000 jobs in the queue during such a run this gets difficult to handle at the time of the crash.

What we've done before (and still do right now) is to mark everything that can't be handled on spot, and do that once the CI environment finishes with the heave load.

With the current load and requirements we might have to change this to be able to handle crashes faster.
I've filed https://github.com/mozilla/mozmill-ci/issues/503 to help us in that regard.

**TLDR: These crashes will be handled and ci mails replied, as always, but we have a slower reaction time due to CI machines much higher load**

Comment 4

4 years ago
(In reply to Henrik Skupin (:whimboo) from comment #1)
> Usually Daniel should have gone through all those reports today, and filed
> bugs appropriately. But sadly I can see your confusion here. There is not a
> single reply on the CI mailing list in case of crashes happened! This is
> bad! I mentioned this a couple of times, especially that this can cause
> confusion for QA people. Now we have such a situation. Please really try to
> get this solved for the duty.
> 

As I already assumpted yesterday on IRC to you, most of the crashes were on mm-osx-106-3, so bug 980800 it's the cause. Machine was busy running tests in the afternoon when the actual tests started. I tried to get the information via ssh, but I hadn't succeed, so I noted in the etherpad that I'll investigate this today.

I did this and my assumption was correct, so I gave the information in bug 980800.

Thanks!

** Oh, it's still today given the different timezone, so I still have time. :)
(In reply to Andrei Eftimie from comment #3)
> To properly investigate (and submit) a crash report we must take the
> affected machine offline, which we can not currently do without disrupting
> the testrun batch. With > 1000 jobs in the queue during such a run this gets
> difficult to handle at the time of the crash.

To mention it again, there is no need to take a machine offline for submitting crash reports. At least not for OS X and Linux. The crash reporter works perfectly from the command line when logged in via SSH. And indeed, I told this on IRC yesterday.

> What we've done before (and still do right now) is to mark everything that
> can't be handled on spot, and do that once the CI environment finishes with
> the heave load.

Please take the above under consideration. Windows might still be an exception, but we might wanna figure out how to get it solved there too.

(In reply to daniel.gherasim from comment #4)
> As I already assumpted yesterday on IRC to you, most of the crashes were on
> mm-osx-106-3, so bug 980800 it's the cause. Machine was busy running tests
> in the afternoon when the actual tests started. I tried to get the
> information via ssh, but I hadn't succeed, so I noted in the etherpad that
> I'll investigate this today.

There was enough information for you to see which test is failing. So this bug could have been created way earlier, and the information posted to the CI list. There is nothing here we would have to wait for. But I have no problems when the work regarding the underlying crash issue is delayed a bit.
Depends on: 980800
I was hoping that the current CI duty would check for those crashes. It didn't happen since Friday. So Cosmin, can you please check the affected boxes from the last comment?
Flags: needinfo?(cosmin.malutan)
Sorry if I missed this bug, none of those crashes was under my watch, but they've been already addressed in bug 980800 comment 22.
All creases are on the same machine mm-osx-106-3 and the same test.
Marking this as dupe of bug 980800.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Flags: needinfo?(cosmin.malutan)
Resolution: --- → DUPLICATE
Duplicate of bug: 980800
Ups sorry I closed this this is actually for tracking the test failure and the other is for tracking the crash in FF.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Mozmill tests have been superseded by Marionette tests.
Status: REOPENED → RESOLVED
Last Resolved: 4 years agoa year ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.