Failure in testAddons_installTheme/test1.js

VERIFIED FIXED

Status

VERIFIED FIXED
7 years ago
6 years ago

People

(Reporter: ashughes, Assigned: vladmaniac)

Tracking

unspecified
Dependency tree / graph

Firefox Tracking Flags

(firefox12 verified, firefox13 verified, firefox14 verified, firefox15 verified, firefox-esr10 verified, status1.9.2 unaffected)

Details

(Whiteboard: [mozmill-test-failure], URL)

Attachments

(2 attachments)

(Reporter)

Description

7 years ago
MODULE: tests/functional/restartTests/testAddons_installTheme
TEST: test1.js
FAIL: TimeoutError("Modal dialog has been found and processed")@resource://mozmill/modules/utils.js:429 waitFor([object Proxy],"Modal dialog has been found and processed",25000,100,[object Proxy])@resource://mozmill/modules/utils.js:467 
BRANCH: mozilla-release, mozilla-beta, mozilla-aurora, default

This failure happens frequently on Windows and sometimes on Linux.
(Reporter)

Comment 1

7 years ago
Please note that test2.js also fails but I suspect it is in response to the above failure.
(Assignee)

Comment 2

7 years ago
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #1)
> Please note that test2.js also fails but I suspect it is in response to the
> above failure.

This failure cannot be fixed in the test for the following reasons: 

1. The exact same error occurs sometimes for all the add-on restart tests since they are handling modal dialogs on installation 
2. It can most certainly be a firefox regression since it can be reproducible sometimes manually both on windows and linux (sent you e-mail some time ago about this, including screencasts) 

Conclusion: 
The investigation will need to focus on: 
- Firefox regression (P1) 
- Handling modal dialogs in our API (P2)
(Reporter)

Comment 3

7 years ago
Given that, we should immediately disable all Addon Restart tests until both issues are corrected. Here is what we should do:

* File a bug for the Firefox regression
* File a bug for the API
* File a bug for each test which needs to be disabled with the above two as blockers
* Disable the tests immediately
* Investigate the regression
* Fix the API
* Re-enable the tests

Thanks
(In reply to Maniac Vlad Florin (:vladmaniac) from comment #2)
> This failure cannot be fixed in the test for the following reasons: 

Not sure why you think that. What gives you the fundamental impression?

> 1. The exact same error occurs sometimes for all the add-on restart tests
> since they are handling modal dialogs on installation 

I don't see why this should be a problem. Are making all those tests use of the local XPI files? Do you have a testcase which is 100% reproducible?

> 2. It can most certainly be a firefox regression since it can be
> reproducible sometimes manually both on windows and linux (sent you e-mail
> some time ago about this, including screencasts) 

What do you mean with 'manual' reproducible?
(Assignee)

Comment 5

7 years ago
(In reply to Henrik Skupin (:whimboo) from comment #4)
> (In reply to Maniac Vlad Florin (:vladmaniac) from comment #2)
> > This failure cannot be fixed in the test for the following reasons: 
> 
> Not sure why you think that. What gives you the fundamental impression?

Investigating test failures procedure implies checking for Firefox regression first

> 
> > 1. The exact same error occurs sometimes for all the add-on restart tests
> > since they are handling modal dialogs on installation 
> 
> I don't see why this should be a problem. Are making all those tests use of
> the local XPI files? Do you have a testcase which is 100% reproducible?

Most of the tests are using local xpi files. There are some exceptions which require remote content (from amo for example)

> 
> > 2. It can most certainly be a firefox regression since it can be
> > reproducible sometimes manually both on windows and linux (sent you e-mail
> > some time ago about this, including screencasts)

> 
> What do you mean with 'manual' reproducible?

Please take a look at this screencast 
http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-b1cc-4073-8ffc-98166b1eae5e

Sometimes, the issue with modal dialog freezing can be manually reproduced.
(Assignee)

Updated

7 years ago
Assignee: nobody → vlad.mozbugs
Status: NEW → ASSIGNED
(Reporter)

Comment 6

7 years ago
(In reply to Maniac Vlad Florin (:vladmaniac) from comment #5)
> Please take a look at this screencast 
> http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-
> b1cc-4073-8ffc-98166b1eae5e
> 
> Sometimes, the issue with modal dialog freezing can be manually reproduced.

I strongly encourage you to file this in a Firefox bug as a blocker to this test and cc Dave Townsend. 

As per our failure policy please write a patch to mark this test disabled immediately.

Thanks
(Assignee)

Updated

7 years ago
Depends on: 702266
(Assignee)

Comment 7

7 years ago
Created attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

Skipping this test for now
Attachment #574299 - Flags: review?(anthony.s.hughes)
(Reporter)

Updated

7 years ago
Attachment #574299 - Flags: review?(anthony.s.hughes) → review+
(Reporter)

Comment 8

7 years ago
Comment on attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

># HG changeset patch
># User Vlad Florin Maniac <vlad.maniac@softvisioninc.eu>
># Date 1321285618 -7200
># Node ID 008c478e747cfd2dda9287311b2ff9dfaac81475
># Parent  aca3662b5c604c34d774e1e16f4b1cc183741c89
>Bug 701893 - Failure in testAddons_installTheme/test1.js, r=ashughes

Just one minor nit which I will fix upon check-in. When disabling a test please word your commit message accordingly.

"Bug 701893 - disable testAddon_installTheme/test1.js due to failure. r=ashughes"
(Assignee)

Comment 9

7 years ago
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #8)
> Comment on attachment 574299 [details] [diff] [review] [diff] [details] [review]
> skip patch v1.0 all branches
> 
> ># HG changeset patch
> ># User Vlad Florin Maniac <vlad.maniac@softvisioninc.eu>
> ># Date 1321285618 -7200
> ># Node ID 008c478e747cfd2dda9287311b2ff9dfaac81475
> ># Parent  aca3662b5c604c34d774e1e16f4b1cc183741c89
> >Bug 701893 - Failure in testAddons_installTheme/test1.js, r=ashughes
> 
> Just one minor nit which I will fix upon check-in. When disabling a test
> please word your commit message accordingly.
> 
> "Bug 701893 - disable testAddon_installTheme/test1.js due to failure.
> r=ashughes"

Thanks
(Reporter)

Comment 10

7 years ago
Comment on attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/0cc6f214904c (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/07072978b714 (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/b69ba4d8e65d (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/66cecab7cb09 (mozilla-release)
Attachment #574299 - Attachment description: skip patch v1.0 all branches → skip patch v1.0 all branches [checked-in]
(Reporter)

Updated

7 years ago
Whiteboard: [mozmill-test-failure] → [mozmill-test-failure][mozmill-test-skipped]
(In reply to Maniac Vlad Florin (:vladmaniac) from comment #5)
> Please take a look at this screencast 
> http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-
> b1cc-4073-8ffc-98166b1eae5e
> 
> Sometimes, the issue with modal dialog freezing can be manually reproduced.

I can't watch because it requires the WMP. Can you instead please describe the steps you have used and at which state the freeze(?) happened for you? Or have you done this already in the filed Firefox bug?
(Assignee)

Comment 12

7 years ago
(In reply to Henrik Skupin (:whimboo) from comment #11)
> (In reply to Maniac Vlad Florin (:vladmaniac) from comment #5)
> > Please take a look at this screencast 
> > http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-
> > b1cc-4073-8ffc-98166b1eae5e
> > 
> > Sometimes, the issue with modal dialog freezing can be manually reproduced.
> 
> I can't watch because it requires the WMP. Can you instead please describe
> the steps you have used and at which state the freeze(?) happened for you?
> Or have you done this already in the filed Firefox bug?

I am currently working with Mossop on identifying the issue, in the filed bug and on e-mails
(Reporter)

Updated

7 years ago
Depends on: 708387
(Assignee)

Updated

7 years ago
Depends on: 710143
(Assignee)

Comment 13

6 years ago
This is the restart test report with the test re-enabled for Ubuntu: its green

http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a18c4fe
Wohoo. Lets get it enabled.
status1.9.2: --- → unaffected
status-firefox-esr10: --- → affected
status-firefox12: --- → affected
status-firefox13: --- → affected
status-firefox14: --- → affected
status-firefox15: --- → affected
(Assignee)

Comment 15

6 years ago
Lets wait for the Mac Os X results 

And for my patch of course :) 
We'll have it in a couple of mins
(Assignee)

Comment 17

6 years ago
Created attachment 624382 [details] [diff] [review]
patch v1.0 reEnable

Obsoleting skip patch
Attachment #574299 - Attachment is obsolete: true
Attachment #624382 - Flags: review?(hskupin)
(Assignee)

Comment 18

6 years ago
We'll need a separate patch for mozilla-esr10 branch but lets land this on default and wait for some results first and if everything is fine on the test machines a patch for esr will follow shortly
Comment on attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

This patch is not obsolete. It has been checked-in.
Attachment #574299 - Attachment is obsolete: false
Attachment #624382 - Flags: review?(hskupin) → review+
Just a note for the next time. If there are multiple patches on a bug please try to find a better commit message which describes what the patch is doing.

Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/f7c0fdbffad7
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Dave, can you please land on older branches if no more failures occur?
status-firefox15: affected → fixed
(In reply to Henrik Skupin (:whimboo) from comment #21)
> Dave, can you please land on older branches if no more failures occur?

Given that it hasn't been done today I have landed those patches now:

http://hg.mozilla.org/qa/mozmill-tests/rev/7fae5006a6e7 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/e16473775823 (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/4b9b72aea131 (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/85b6d75eb913 (esr10)

Vlad please update all Litmus tests for the Mozmill subgroups and information, and link those here. Thanks.
status-firefox-esr10: affected → fixed
status-firefox12: affected → fixed
status-firefox13: affected → fixed
status-firefox14: affected → fixed
Whiteboard: [mozmill-test-failure][mozmill-test-skipped] → [mozmill-test-failure]
(In reply to Maniac Vlad Florin (:vladmaniac) from comment #23)
> Litmus tests updated and added to Mozmill subgroups:

This also needs an update for the Firefox 10 Litmus test. Please closely check the status flags when doing those updates.
Thanks.
(Reporter)

Comment 27

6 years ago
No recent reports of this failure, marking verified.
Status: RESOLVED → VERIFIED
status-firefox-esr10: fixed → verified
status-firefox12: fixed → verified
status-firefox13: fixed → verified
status-firefox14: fixed → verified
status-firefox15: fixed → verified
You need to log in before you can comment on or make changes to this bug.