701893 - Failure in testAddons_installTheme/test1.js

Status: VERIFIED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[u279076]

MODULE: tests/functional/restartTests/testAddons_installTheme
TEST: test1.js
FAIL: TimeoutError("Modal dialog has been found and processed")@resource://mozmill/modules/utils.js:429 waitFor([object Proxy],"Modal dialog has been found and processed",25000,100,[object Proxy])@resource://mozmill/modules/utils.js:467 
BRANCH: mozilla-release, mozilla-beta, mozilla-aurora, default

This failure happens frequently on Windows and sometimes on Linux.

Comment 1

[u279076]

Please note that test2.js also fails but I suspect it is in response to the above failure.

Comment 2

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #1)
> Please note that test2.js also fails but I suspect it is in response to the
> above failure.

This failure cannot be fixed in the test for the following reasons: 

1. The exact same error occurs sometimes for all the add-on restart tests since they are handling modal dialogs on installation 
2. It can most certainly be a firefox regression since it can be reproducible sometimes manually both on windows and linux (sent you e-mail some time ago about this, including screencasts) 

Conclusion: 
The investigation will need to focus on: 
- Firefox regression (P1) 
- Handling modal dialogs in our API (P2)

Comment 3

[u279076]

Given that, we should immediately disable all Addon Restart tests until both issues are corrected. Here is what we should do:

* File a bug for the Firefox regression
* File a bug for the API
* File a bug for each test which needs to be disabled with the above two as blockers
* Disable the tests immediately
* Investigate the regression
* Fix the API
* Re-enable the tests

Thanks

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Maniac Vlad Florin (:vladmaniac) from comment #2)
> This failure cannot be fixed in the test for the following reasons: 

Not sure why you think that. What gives you the fundamental impression?

> 1. The exact same error occurs sometimes for all the add-on restart tests
> since they are handling modal dialogs on installation 

I don't see why this should be a problem. Are making all those tests use of the local XPI files? Do you have a testcase which is 100% reproducible?

> 2. It can most certainly be a firefox regression since it can be
> reproducible sometimes manually both on windows and linux (sent you e-mail
> some time ago about this, including screencasts) 

What do you mean with 'manual' reproducible?

Comment 5

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Henrik Skupin (:whimboo) from comment #4)
> (In reply to Maniac Vlad Florin (:vladmaniac) from comment #2)
> > This failure cannot be fixed in the test for the following reasons: 
> 
> Not sure why you think that. What gives you the fundamental impression?

Investigating test failures procedure implies checking for Firefox regression first

> 
> > 1. The exact same error occurs sometimes for all the add-on restart tests
> > since they are handling modal dialogs on installation 
> 
> I don't see why this should be a problem. Are making all those tests use of
> the local XPI files? Do you have a testcase which is 100% reproducible?

Most of the tests are using local xpi files. There are some exceptions which require remote content (from amo for example)

> 
> > 2. It can most certainly be a firefox regression since it can be
> > reproducible sometimes manually both on windows and linux (sent you e-mail
> > some time ago about this, including screencasts)

> 
> What do you mean with 'manual' reproducible?

Please take a look at this screencast 
http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-b1cc-4073-8ffc-98166b1eae5e

Sometimes, the issue with modal dialog freezing can be manually reproduced.

Comment 6

[u279076]

(In reply to Maniac Vlad Florin (:vladmaniac) from comment #5)
> Please take a look at this screencast 
> http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-
> b1cc-4073-8ffc-98166b1eae5e
> 
> Sometimes, the issue with modal dialog freezing can be manually reproduced.

I strongly encourage you to file this in a Firefox bug as a blocker to this test and cc Dave Townsend. 

As per our failure policy please write a patch to mark this test disabled immediately.

Thanks

Comment 7

[Maniac Vlad Florin (:vladmaniac)]

Attachment: skip patch v1.0 all branches [checked-in]

Skipping this test for now

Comment 8

[u279076]

Comment on attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

># HG changeset patch
># User Vlad Florin Maniac <[e-mail]>
># Date 1321285618 -7200
># Node ID 008c478e747cfd2dda9287311b2ff9dfaac81475
># Parent  aca3662b5c604c34d774e1e16f4b1cc183741c89
>Bug 701893 - Failure in testAddons_installTheme/test1.js, r=ashughes

Just one minor nit which I will fix upon check-in. When disabling a test please word your commit message accordingly.

"Bug 701893 - disable testAddon_installTheme/test1.js due to failure. r=ashughes"

Comment 9

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #8)
> Comment on attachment 574299 [details] [diff] [review] [diff] [details] [review]
> skip patch v1.0 all branches
> 
> ># HG changeset patch
> ># User Vlad Florin Maniac <[e-mail]>
> ># Date 1321285618 -7200
> ># Node ID 008c478e747cfd2dda9287311b2ff9dfaac81475
> ># Parent  aca3662b5c604c34d774e1e16f4b1cc183741c89
> >Bug 701893 - Failure in testAddons_installTheme/test1.js, r=ashughes
> 
> Just one minor nit which I will fix upon check-in. When disabling a test
> please word your commit message accordingly.
> 
> "Bug 701893 - disable testAddon_installTheme/test1.js due to failure.
> r=ashughes"

Thanks

Comment 10

[u279076]

Comment on attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/0cc6f214904c (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/07072978b714 (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/b69ba4d8e65d (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/66cecab7cb09 (mozilla-release)

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Maniac Vlad Florin (:vladmaniac) from comment #5)
> Please take a look at this screencast 
> http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-
> b1cc-4073-8ffc-98166b1eae5e
> 
> Sometimes, the issue with modal dialog freezing can be manually reproduced.

I can't watch because it requires the WMP. Can you instead please describe the steps you have used and at which state the freeze(?) happened for you? Or have you done this already in the filed Firefox bug?

Comment 12

[Maniac Vlad Florin (:vladmaniac)]

(In reply to Henrik Skupin (:whimboo) from comment #11)
> (In reply to Maniac Vlad Florin (:vladmaniac) from comment #5)
> > Please take a look at this screencast 
> > http://www.screencast.com/users/NoIdea1234/folders/Default/media/ed4b1cdd-
> > b1cc-4073-8ffc-98166b1eae5e
> > 
> > Sometimes, the issue with modal dialog freezing can be manually reproduced.
> 
> I can't watch because it requires the WMP. Can you instead please describe
> the steps you have used and at which state the freeze(?) happened for you?
> Or have you done this already in the filed Firefox bug?

I am currently working with Mossop on identifying the issue, in the filed bug and on e-mails

Comment 13

[Maniac Vlad Florin (:vladmaniac)]

This is the restart test report with the test re-enabled for Ubuntu: its green

http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a18c4fe

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Wohoo. Lets get it enabled.

Comment 15

[Maniac Vlad Florin (:vladmaniac)]

Lets wait for the Mac Os X results 

And for my patch of course :) 
We'll have it in a couple of mins

Comment 16

[Maniac Vlad Florin (:vladmaniac)]

Mac is also green 

http://mozmill-crowd.blargon7.com/#/functional/report/f87375a634b1a5ba746e5f763a19506a

Comment 17

[Maniac Vlad Florin (:vladmaniac)]

Attachment: patch v1.0 reEnable

Obsoleting skip patch

Comment 18

[Maniac Vlad Florin (:vladmaniac)]

We'll need a separate patch for mozilla-esr10 branch but lets land this on default and wait for some results first and if everything is fine on the test machines a patch for esr will follow shortly

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 574299 [details] [diff] [review]
skip patch v1.0 all branches [checked-in]

This patch is not obsolete. It has been checked-in.

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Just a note for the next time. If there are multiple patches on a bug please try to find a better commit message which describes what the patch is doing.

Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/f7c0fdbffad7

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dave, can you please land on older branches if no more failures occur?

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Henrik Skupin (:whimboo) from comment #21)
> Dave, can you please land on older branches if no more failures occur?

Given that it hasn't been done today I have landed those patches now:

http://hg.mozilla.org/qa/mozmill-tests/rev/7fae5006a6e7 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/e16473775823 (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/4b9b72aea131 (release)
http://hg.mozilla.org/qa/mozmill-tests/rev/85b6d75eb913 (esr10)

Vlad please update all Litmus tests for the Mozmill subgroups and information, and link those here. Thanks.

Comment 23

[Maniac Vlad Florin (:vladmaniac)]

Litmus tests updated and added to Mozmill subgroups:

https://litmus.mozilla.org/show_test.cgi?id=15439
https://litmus.mozilla.org/show_test.cgi?id=50562
https://litmus.mozilla.org/show_test.cgi?id=55432

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Maniac Vlad Florin (:vladmaniac) from comment #23)
> Litmus tests updated and added to Mozmill subgroups:

This also needs an update for the Firefox 10 Litmus test. Please closely check the status flags when doing those updates.

Comment 25

[Maniac Vlad Florin (:vladmaniac)]

Done! Firefox 10: https://litmus.mozilla.org/show_test.cgi?id=40537

Comment 26

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thanks.

Comment 27

[u279076]

No recent reports of this failure, marking verified.