Closed Bug 809124 Opened 12 years ago Closed 11 years ago

restartTests/testAddons started failing with "Modal dialog has been found and processed" on 11-05-2012

Categories

(Mozilla QA Graveyard :: Mozmill Tests, defect, P3)

Product:
Mozilla QA Graveyard
Component:
Mozmill Tests
Type:
defect

Tracking

(firefox19 fixed, firefox20 fixed, firefox21 fixed, firefox22 fixed, firefox-esr17 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox19 --- fixed
firefox20 --- fixed
firefox21 --- fixed
firefox22 --- fixed
firefox-esr17 --- fixed

People

(Reporter: AlexLakatos, Assigned: mario.garbi)

Details

(Whiteboard: [mozmill-test-failure] s=130204 u=failure c=addons p=1)

Attachments

(1 file)

patch v1.0
21.57 KB, patch
AndreeaMatei
: review+
Details | Diff | Splinter Review 
TEST: /restartTests/testAddons_enableDisableExtension/test1.js | test1.js::testInstallAddon
TEST: /restartTests/testAddons_installMultipleExtensions/test1.js | test1.js::testInstallMultipleExtensions
TEST: /restartTests/testAddons_installFromFTP/test1.js | test1.js::testInstallAddonFromFTP
ERROR: Modal dialog has been found and processed
WHEN: 11-05-2012
FIRST: 11-05-2012
FREQ: 1

mozmill-ci reports:

http://mozmill-ci.blargon7.com/#/functional/failure?branch=All&platform=All&from=2012-11-01&&test=%2FrestartTests%2FtestAddons_installFromFTP%2Ftest1.js&func=test1.js%3A%3AtestInstallAddonFromFTP

http://mozmill-ci.blargon7.com/#/functional/failure?branch=All&platform=All&from=2012-11-01&to=&test=%2FrestartTests%2FtestAddons_installMultipleExtensions%2Ftest1.js&func=test1.js%3A%3AtestInstallMultipleExtensions

http://mozmill-ci.blargon7.com/#/functional/failure?branch=All&platform=All&from=2012-11-01&to=&test=%2FrestartTests%2FtestAddons_enableDisableExtension%2Ftest1.js&func=test1.js%3A%3AtestInstallAddon
Alex, your initial comment is not that helpful. Please stick to the template we have created for new test failures. On which branches can this problem be seen? Which platforms are affected? All that has not been mentioned.
Priority: -- → P1
As it can be seen from the reports, tests that install addons started failing on nightly, across different Windows platforms.
status-firefox19: --- → affected
Then it's not platform 'All'. Are you able to reproduce it locally?
OS: All → Windows 8
Again on testAddons_installFromFTP, with beta this time - Windows XP:
http://mozmill-ci.blargon7.com/#/functional/report/a11aa7b15f4e571fd6fe21a2db1e4f0d
Can we please re-run the 18.0 beta job in CI? I would like to see if it could have been due to some load issues.
No failures in the last month. Closing out as WFM for now.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WORKSFORME
Happened again in 2013-02-03 on Mac OS X 10.7.5 (x86_64) - Firefox 21.0a1:
http://mozmill-ci.blargon7.com/#/functional/report/0dbaa964aec88a2f5637c68c968a8f7f
Please do not forget to reopen a bug if it happens again and set the appropriate flags. Thanks.
Status: RESOLVED → REOPENED
status-firefox21: --- → affected
OS: Windows 8 → All
Priority: P1 → P3
Resolution: WORKSFORME → ---
Depends on: 826693
The modal dialog failures happen because Firefox pings AMO for compatibility updates when installing an extension or theme. If AMO doesn't respond our tests will fail even when we install the addon from localhost. We might want to stop those checks.

Joel gave me a link via IRC how other automation is working around this problem:
http://mxr.mozilla.org/mozilla-central/source/build/automation.py.in#511

We might want to leave those checks enabled in the remote tests.
Status: REOPENED → NEW
No longer depends on: 826693
Whiteboard: [mozmill-test-failure] → [mozmill-test-failure] s=130204 u=failure c=addons p=1
Assignee: nobody → mario.garbi
(In reply to Henrik Skupin (:whimboo) [away 02/09 - 02/017] from comment #10)
> The modal dialog failures happen because Firefox pings AMO for compatibility
> updates when installing an extension or theme. If AMO doesn't respond our
> tests will fail even when we install the addon from localhost. We might want
> to stop those checks.

Are we sure this is the case?

I was trying to reproduce this issue, but all (not sure if I ran *all* of them, but a good batch) local Addons tests passed  with no network connectivity, which means that we do not really ping AMO at the moment.
Maybe I am misunderstanding how this should work, but shouldn't our tests fail in this case?

Does anyone have more info about the AMO compatibility ping?
(In reply to Andrei Eftimie from comment #11)
> (In reply to Henrik Skupin (:whimboo) [away 02/09 - 02/017] from comment #10)
> > The modal dialog failures happen because Firefox pings AMO for compatibility
> > updates when installing an extension or theme. If AMO doesn't respond our
> > tests will fail even when we install the addon from localhost. We might want
> > to stop those checks.
> 
> Are we sure this is the case?
> 
> I was trying to reproduce this issue, but all (not sure if I ran *all* of
> them, but a good batch) local Addons tests passed  with no network
> connectivity, which means that we do not really ping AMO at the moment.
> Maybe I am misunderstanding how this should work, but shouldn't our tests
> fail in this case?
> 
> Does anyone have more info about the AMO compatibility ping?

What do you mean by 'no network connectivity'? If you mean no network connection at all, then Firefox may behave differently in this case. To reproduce this issue you may need to redirect the pings to a server that will not respond appropriately.
I've made 2 different attempts.

1. ifconfig en0 down
(only wired network)
tests went fine

2.
a) used a proxy to see network traffic while running tests
b) edited hosts file
blocked the following addresses:
127.0.0.1 services.addons.mozilla.org
127.0.0.1 versioncheck.addons.mozilla.org
127.0.0.1 versioncheck-bg.addons.mozilla.org
127.0.0.1 pyrepo.addons.mozilla.org

Tests still ran fine.
============

Wasn't the problem reported as being a network issue between where our test machines are and where AMO located is?
If that is indeed the case a local DNS redirect should do the trick...
(In reply to Andrei Eftimie from comment #13)
> 1. ifconfig en0 down
> (only wired network)
> tests went fine

If there is no network connection no compatibility check will be performed.

> 2.
> a) used a proxy to see network traffic while running tests
> b) edited hosts file
> blocked the following addresses:
> 127.0.0.1 services.addons.mozilla.org
> 127.0.0.1 versioncheck.addons.mozilla.org
> 127.0.0.1 versioncheck-bg.addons.mozilla.org
> 127.0.0.1 pyrepo.addons.mozilla.org
> 
> Tests still ran fine.

Correct, and that's what I expect. You have blocked those servers which means we can't reach the hosts for a compatibility check. That means we don't do a check; same as above.

> Wasn't the problem reported as being a network issue between where our test
> machines are and where AMO located is?
> If that is indeed the case a local DNS redirect should do the trick...

Network issue doesn't mean that there is no network at all. The site was not responding to requests and so we timed out in the installation dialog while the compatibility check was running.

If your proxy can slow down the response you should be able to replicate this issue. So I still would like to see that we disable such a check.
Attached patch patch v1.0 — — Splinter Review 
With this patch we don't have the compatibility check, as instructed by Henrik in comment 10.
 I wasn't able to reproduce it locally but I will get a proxy on Monday and will come back with reports.
Attachment #719967 - Flags: review?
this attachment has a r? with nobody assigned to review it.
Comment on attachment 719967 [details] [diff] [review]
patch v1.0

I'm sorry for not selecting a reviewer in the first place, and thank you Joel Maher for pointing it out.
Attachment #719967 - Flags: review? → review?(andreea.matei)
Comment on attachment 719967 [details] [diff] [review]
patch v1.0

Review of attachment 719967 [details] [diff] [review]:
-----------------------------------------------------------------

Please use capital letters for commit message and make sure that it's one relevant to your changes.
I have change those this time.

Lets see how this works, pushed to default:
http://hg.mozilla.org/qa/mozmill-tests/rev/1d4e86b6d128

I'm curious to see an update and some reports when you are able to reproduce the issue.
Attachment #719967 - Flags: review?(andreea.matei) → review+
 Due to a patch that landed yesterday this one will not apply cleanly anymore, I will come with reports and an updated patch asap.
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: